Moviepy Roadmap - version 2.0

[tburrows13]

I’d like to propose an upcoming v2.0 which would allow us the opportunity to make a few small breaking changes, and remove some of the previously deprecated features.

Here’s some of the things that I’d like to include:

• [Start v2.0 branch. Remove support for Python 2.7 & 3.4. #1103, Remove support for Python 3.5 #1106] Remove support for old Python versions (Drop support for python < 3.6 #1081)

  • Definitely 2.7 and 3.4, perhaps 3.5 as well?
  • Py3.5 is not EOL yet, accounts for about 2% of our installs (https://pypistats.org/packages/moviepy), and the number of new features in Py3.6 is very small (but it might be worth it just for the f-strings)
  • This would allow us to get rid of a lot of complicated and hacky code

• I’d like to go through the backlog of complaints about resource usage and proper closing of clips and create a consistent cross-platform solution that can work for everyone, using some to-be-decided combination of .close(), .__del__() and context managers (with ...:). This would most likely not be backwards-compatible (Fixed problems with proc package #1016, Don't close the audio reader which might be linked from a shallow copy. #1059).

• We could enforce use of “keyword-only” arguments with the * (https://www.python.org/dev/peps/pep-3102/)

  • For instance, the definition of VideoFileClip would become

    def __init__(self, filename, *, has_mask=False,
         audio=True, audio_buffersize=200000,
         target_resolution=None, resize_algorithm='bicubic',
         audio_fps=44100, audio_nbytes=2, verbose=False,
         fps_source='tbr'):
    

  • So VideoFileClip("file.mp4", True, True, 100000) would now be invalid

• If it were possible to get rid of ImageMagick as a dependency, that would be amazing, its possible that everything that we use it for can now be done with imageio or one of our other, better behaved, dependancies. (Discussion in Imageio's new use of imageio-ffmpeg #908, TextClip produces low quality text #1472)

• There are a lot of functions/methods with lots of optional parameters like https://github.com/Zulko/moviepy/blob/master/moviepy/video/fx/crop.py that require certain combinations of arguments to work. It would be nice to have a neat way of expressing this, maybe with decorators?

• [Remove deprecated functions and parameters #1115] We can remove old deprecations

  • to_videofile -> write_videofile
  • to_audiofile -> write_audiofile
  • ColorClip(col=...)

• [Renaming methods, parameters and variables for increased usability, consistency and readability #1170] We could go even further and improve the API even more

  • set_duration -> with_duration etc is a common sticking point for new users not realising that it doesn't act inplace (TextClip positioning does not work #886) (or do what pandas does and have an inplace=False parameter to everything, possibly handled by a decorator)
  • Similar improvements to the set_start (set_start and set_end have no effect other than changing the clip's duration #995)

There are other things that need to be done, that, whilst they could be released at any time, would be good to include as part of a larger package

• Implement basic data model #1076 implements support for built-in Python operators
• I'm working on improving the install/test/build/release system
• Updated/checked examples would be great
• Generally the documentation is lacking in some areas; at a minimum it should state every parameter that a function accepts
• Any other small features (Add support for path-like objects #924...)

Since there's a lot of changes here, I'd propose merging them into a v2 branch first, before finally merging that into master when we're ready to release.

Anything else that you can think of, agree/disagree with, let's discuss either here or on gitter!

[misland]

hi,do you hava a plan for this package to make it support GPU?

[tburrows13]

@misland Thanks for the suggestion, I'll add it to the list. There's clearly interest (#790, #923), and it looks like it would be just a case of configuring FFmpeg settings.

[misland]

@tburrows13 thanks for your reply,I also checked #790 and I will test ffmpeg which compiled with gpu library later,I will post the result when I finish testing.

[misland]

@tburrows13 hi,I modify the ffmpeg_reader.py as follows:

I execute one python script to process mp4 video,then I do see two processors by nvidia-smi,as follows:

it seems to worked,but the cpu cost didn't go down at all,so I don't think gpu acceleration take effect,hope it helps to you :(

[tburrows13]

@misland, thanks for investigating. I'll have a look at some point to see if it would be benificial, but it doesn't seem to require much extra code on our part, so it will probably end up being an option for end users to use if they want to.

[tburrows13]

I don't think that having v2 as a separate branch is really working, so I'll release v1.0.2 today, and then merge master into v2, then v2 back into master, so that new contributions will automatically be based off the v2 code.

[tburrows13]

I don't think that having v2 as a separate branch is really working, so I'll release v1.0.2 today, and then merge master into v2, then v2 back into master, so that new contributions will automatically be based off the v2 code.

Now done, so all v2 code is in the current master branch.

[mgaitan]

Oh, didn't see these last comment. I've been opening the prs against v2 :-)

[tburrows13]

Ok, I'll delete v2 now as it doesn't seem to have any purpose.

[tburrows13]

I thought that it would be a good idea to post my current thoughts regarding the v2.0 release (for my own sake as much as anything else!). There are a few outstanding major PRs that are still awaiting merges and there are a few more things that don't yet have a PR that I am still planning to do.

Current PRs:

• Implement basic data model #1076 Implement basic data model - requires further discussion
• Change ways to composite video clips by using PIL #1157 Optimise CompositeVideoClip by using PIL
• Renaming methods, parameters and variables for increased usability, consistency and readability #1170 Renaming methods, parameters and variables for increased usability, consistency and readability - requires further thought/discussion
• Fix support for grayscale images (there are already some PRs about this)

Other:

• Improve imports, including Fluent API for effects (monkeypatch as methods) #1105, but also my comments there as a separate PR
• Rewrite TextClip to use pillow (discussed and partially implemented in Some advice for improve CompositeVideoClip TextClip and the blit #1145)
• Improve preview(), perhaps using VLC's python bindings
• Fix some fx bugs that I found (loop and resize with lambda functions)
• More tests!

Documentation:

• Comb through every docstring to ensure that all parameters are explained and formatting is consistent
• Create an improved 'Getting Started' tutorial
• Ensure that the examples are all functional and useful. Add more examples
• Move documentation to readthedocs.org. This allows it to be automatically built whenever an update is posted, and they can store different versions of the documentation for different git tagged versions.

I'm very happy to discuss any of these plans before actually implementing them!

[mike592]

gpu

do you mean GPU don't help to make moviepy do rendering jobs faster at all?

[averypfeiffer]

I thought that it would be a good idea to post my current thoughts regarding the v2.0 release (for my own sake as much as anything else!). There are a few outstanding major PRs that are still awaiting merges and there are a few more things that don't yet have a PR that I am still planning to do.

Current PRs:

• Implement basic data model #1076 Implement basic data model - requires further discussion
• Change ways to composite video clips by using PIL #1157 Optimise CompositeVideoClip by using PIL
• Renaming methods, parameters and variables for increased usability, consistency and readability #1170 Renaming methods, parameters and variables for increased usability, consistency and readability - requires further thought/discussion
• Fix support for grayscale images (there are already some PRs about this)

Other:

• Improve imports, including Fluent API for effects (monkeypatch as methods) #1105, but also my comments there as a separate PR
• Rewrite TextClip to use pillow (discussed and partially implemented in Some advice for improve CompositeVideoClip TextClip and the blit #1145)
• Improve preview(), perhaps using VLC's python bindings
• Fix some fx bugs that I found (loop and resize with lambda functions)
• More tests!

Documentation:

• Comb through every docstring to ensure that all parameters are explained and formatting is consistent
• Create an improved 'Getting Started' tutorial
• Ensure that the examples are all functional and useful. Add more examples
• Move documentation to readthedocs.org. This allows it to be automatically built whenever an update is posted, and they can store different versions of the documentation for different git tagged versions.

I'm very happy to discuss any of these plans before actually implementing them!

Love to see this project moving forward. I don't see GPU acceleration in the progress update above, is this still on your radar? Would be incredibly useful in a lot of use-cases. I don't mind volunteering some time to help solve the GPU issue.

[tburrows13]

Hello, thanks for your interest! Here's where I'm up to:

I'll have some time over the next 4-5 weeks to work on moviepy and hopefully I'll be able to push v2.0 at the end of that.
My 5 priorities are:

• Renaming methods, parameters and variables for increased usability, consistency and readability #1170 - mostly done, there's a bit more to do in a separate PR: the big rename. Basically ready to go, but I'll need to fix quite a few merge conflicts that have arisen since I created it. If you have suggestions on API changes (in particular the ones that I've made there), I'd appreciate feedback.

• Corrections to video duration detection #1222 and associated: sorting out FFmpeg duration/frame rate detection. Its a pain, I've asked on the FFmpeg mailing list and the conclusion is that there's not much that can be done about it, so I think that I am currently planning to revert back to the pre-2.0 way of doing it with an option to fully decode the video file (slow for large videos but presumably more accurate)

• Fluent API for effects (monkeypatch as methods) #1105 + Move most imports from editor.py to __init__.py #1340: big import change. Should be straightforward as Fluent API for effects (monkeypatch as methods) #1105 already has most of the work done

• Implement basic data model #1076 the clip 'data' model. It is complete but there hasn't been a resolution on which bits of it should actually be merged. Input would be appreciated!

• Some advice for improve CompositeVideoClip TextClip and the blit #1145 finish alternative TextClip without ImageMagick dependency

---

As well as those 'big' items, I am planning to rework the documentation. The examples and tutorials are very outdated and lots of parameters are poorly documented or not mentioned at all. In addition, I'd like to move the docs to readthedocs.io, because they automatically update the docs on every push and they add a dropdown which allows the user to switch between docs for different versions. This allows future documentation improvements to take effect in realtime.

When all of the above is done, I'd like to release it as 2.0.0. At various stages throughout it I'll probably release more dev releases (we already have v2.0.0.dev1).

---

Here's where anyone/everyone else comes in :)
The v2.0.0 milestone has a list of issues and PRs that I've previously thought would be good to be in v2.0. If you're wanting something to do, that would be a good place to start! I'm not expecting all of them to actually be done by v2.0 by any means.

You mentioned GPU acceleration: there have been a lot of requests for either that or multi-threading, because moviepy is quite slow! They would be great features to have, however I expect that they'd be quite complicated but also quite backwards-compatible so I see them as something that I'd probably work on for 2.1. However, if anyone else would like to do them and they are ready in time for 2.0, I'd happily add them in. #698 has a proof of concept for multi-threading.

If you do decide to work on GPU/multi-threading, let me know what you are working on and I may be able to help :)

[keikoro]

A looong, long time ago, I updated and fixed parts of the docs, but the changes were ultimately rejected because I combined code + doc fixes. I'll try to dig them up to see if it would make sense to reuse them (or parts of them).

[keikoro]

Ok, I checked, and it looks like the changes I suggested back then were in fact merged in. (They just aren't labelled as mine because they happened within a PR.)

[dheerajmpai]

I guess GPU support is one of the key feature with popular demand. It is preferred this is added in the settings file itself

[hpp0hpp]

I would ask again for the GPU supporting for write_videofile, I see lots of demanding for this. Thank you

[cryzed]

I suppose this is somewhat of a niche request, but since we are suggesting bigger changes we might want to see in 2.0, could I suggest changing the ffmpeg instance handling? For every VideoClipFile instance, I think one or two ffmpeg instances are spawned and running in the background while the videos aren't close()d. Now I recently wanted to combine several hundred videos into one and quickly ran into issues due to the hundreds of running ffmpeg instances. I had to write special handling that would create certain parts of the final video one at a time, taking care to never open more than n ffmpeg instances to not crash my PC. Another drawback is that I wanted to avoid re-encoding the partial clips during the rendering of the final clip, so I had to save those files to disk using codec="png" temporarily, which results in 3x 30gb files in my case.

Would something like a "lazy" mode work, where every time the user queries data from the video, the ffmpeg instance is spun up, queried and then closed immediately? For certain scripts this would be much slower of course, but if it could be turned on selectively per VideoClipFile instance, I think it would allow working with much larger volumes of files at once without running into memory issues. If the actual process of merging many clips into one, using concatenate_videoclips() etc., doesn't require all files to be loaded into memory at once, but only for example 2, the memory consumption could be much lower.

[tburrows13]

@cryzed that seems like a good idea, and I think that it would be feasible. However, I don't think that that would be a breaking change, so I wouldn't neccasarily think it should go in v2.0. I'll continue to consider it though!

[cryzed]

@tburrows13 thank you for the quick reply, and yes I might have been a bit overeager in commenting it on this issue! Sorry about that :). I'm glad to hear you are considering it.

[dheerajmpai]

@cryzed I feel you should look for the potential speedup GPU can provide. GpU does not necessarily speed up rendering. In my case it is around 1.3x speed up and not much.

[cryzed]

@dheerajmpai the speed is fine, that's not the problem. The problem is memory utilization. If I want to open 500 small files at once, 500 ffmpeg instances have to run in the background during the lifetime of the objects.

[fortunto2]

Interesting implementation of using ffmpeg and a video card gpu in this library https://github.com/dmlc/decord

[pingvinton]

In my case I try add '-hwaccel', 'cuda', '-c:v', 'h264_cuvid' params for read and write VideoFileClip and don't get any performance improvements. As result for reading I have started FFmpeg with this params:
FFMmpeg params: ['ffmpeg', '-hwaccel', 'cuda', '-c:v', 'h264_cuvid', '-i', './videos/file.mp4', '-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=1920:1080', '-sws_flags', 'bicubic', '-pix_fmt', 'rgb24', '-vcodec', 'rawvideo', '-']

I don't know why adding cuda acceleration doesn't improve performance.
I tested it on GeForce RTX 3090

Thanks.

[keikoro]

@pingvinton Did you mean to comment on this issue? It's specifically about progressing Moviepy as a library and your comment seems unrelated. Please open a separate, new issue in which you describe your problem in detail.

[owocado]

Hi, are there any plans to release 2.0.0.dev3 on pypi? 2.0.0.dev2 was released in October 2020 and there is changes on master branch since then. 😅

[B3QL]

Hi, are there any plans to release 2.0.0.dev3 on pypi? 2.0.0.dev2 was released in October 2020 and there is changes on master branch since then. sweat_smile

Accordingly to #1768 (comment) it should be soon :) but AFAIK the #1076 PR needs to be merged before and it waits for fixes from the original maintainer.

@tburrows13 is there any plan for moving the project closer to 2.0 release?

[kskadart]

Hey @averypfeiffer
I want to try implement a GPU acceleration, can we have a short discussion?

[arnavmehta7]

Me too!

[yemibox51]

Is there a discord or communication channel we can use to communicate potential solutions and their trade-offs to issues, particularly if there is someone who knows why a particular thing was written a certain way or if they know about it?

[keikoro]

There's a Gitter (see link in README) but it's not really active. There is no other chat-like solution currently.

[tburrows13]

https://twitter.com/gdb/status/1638971232443076609 apparently ChatGPT likes to use moviepy...?!