[Windows] High memory usage when opening many videos to concatenate them

[gam-354]

Hi!

I have been using MoviePy to concatenate many (2000+) videos from a recent trip to Japan. One issue that I have found is that the memory usage grew a lot while I was creating VideoFileClip objects from the video files, prior to passing the list to concatenate_videoclips() function. I didn't need to make any change in the videos, apart from resizing some of them.

Using the Windows Task Manager I found that the memory usage was in fact coming from many ffmpeg processes, as one ffmpeg "Popen" call is made for each of the videos. After about 100 video clips loaded, my 8 GB of memory were gone and my script crashed.

Looking at the code, I found that the FFMPEG_VideoReader::get_frame() method is able to detect if the proc is open, taking care of opening it if it is closed. That gave me the idea that I could keep my clips closed() until the final CompositeVideoClip::write_videofile() call is made. Inside there, the get_frame() call of each clip would be made and the ffmpegs instances would be instantiated when really needed.

To test this, I just created the following method in VideoFileClip so that the ffmpeg reader is closed but not deleted:

    def close_video_reader(self):
        if self.reader:
            self.reader.close()

And I call it right after creating each of my 2000 VideoFileClip objects. This seems to be enough to be able to concatenate some hundreds of video clips. I would expect to have moved the memory "leak" to the write_videofile() call, but surprisingly, it works and stays in 2-3 GB of memory usage.

I start this discussion since I think I have solved my problem, but I feel that it might be a hack rather than a fix to MoviePy. Maybe I am using it wrongly? Would it be interesting to consider it to be a functionality? From an implementation-agnostic point of view, it makes sense to keep the memory free until the video encoding really needs it. From an implementation-wise point of view, in fact, as all Clips implement the is_playing() method, it would be quite easy to detect when a clip is no longer playing to proceed closing the ffmpeg reader.

Feedback is welcome.
Thank you for this project!