Add configuration to set how many threads each ffmpeg process can use

[KyleMaas]

Description

I was running into issues where, while transcoding, a single server would get a load average of over 4x the number of cores it had. This turned out to be a combination of Celery running as many tasks concurrently as it could and each ffmpeg process using all cores. This would result in a ton of chunks' transcoding processes competing for CPU, which would cause them to time out. This change allows you to at least limit the number of threads that ffmpeg can use concurrently so you can actually tune transcoding CPU usage properly.

Steps

Pre-deploy

Post-deploy

[tobocop2]

I noticed this as well, I think this should be merged in as well @mgogoulos

[tobocop2]

@KyleMaas i think this threads setting should also be applied to the

produce_sprite_from_video and produce_thumbnails_from_video functions.

and this block

    if profile.extension == "gif":
        tf = create_temp_file(suffix=".gif")
        # -ss 5 start from 5 second. -t 25 until 25 sec
        command = [
            settings.FFMPEG_COMMAND,
            "-y",
            "-ss",
            "3",
            "-i",
            media.media_file.path,
            "-hide_banner",
            "-vf",
            "scale=344:-1:flags=lanczos,fps=1",
            "-t",
            "25",
            "-f",
            "gif",
            tf,
        ]

in tasks.py

These commands are vulnerable to the same problem.

[KyleMaas]

@tobocop2 Good idea. Implemented.

[dogweather]

Ping