You should add stop -or- reset method (Javascript). Lightbox is only a trick of hide/show some overlay content (This is not stooping your video).
“How to” - its more a task for freelancer - you should know a little JS to solve this:
(You find examples of “how to stop 1video” on click on button X - in your case is a list of videos).
Related (I think you wont find copy-paste answer under this Q - but this is good start):
https://discourse.webflow.com/t/how-to-pause-and-stop-a-video-in-a-modal-on-close/2366/22
https://discourse.webflow.com/search?q=stop%20video%20modal
Change the subject of your Q to “custom code”.