[HOW-TO] Dropping frames when recording w/ picamera2, but not with libcamera-vid (1080p60)

[Phooood]

I'm running a Pi Zero 2 W, overclocked. I can consistently get a nice, smooth 1080p60 video out of it using:
libcamera-vid --frames 300 -b 5000000 --width 1920 --height 1080 --framerate 60 -o test.h264 --level 4.2 --denoise cdn_off

The end goal is essentially a dashcam for a rocket, where the Pi is recording and deleting videos at timed intervals until it detects a launch, where it will continue recording until it lands, then save said recording. It needs to be small & light (and not cost a small fortune), which is why I've chosen a Pi Zero.

The video I'm getting out of my program is sped up (like the Pi isn't fast enough), though it is 60fps, which I assume means I'm dropping frames.

Here is my configuration within the program:

ctrls = cam.video_configuration

#1080p60
ctrls.controls.FrameRate = 60
ctrls.size = (1920, 1080)
ctrls.buffer_count = 10
ctrls.controls.NoiseReductionMode = 1
ctrls.format = 'YUV420'
ctrls.controls.AfMode = controls.AfModeEnum.Continuous

encoder = H264Encoder(framerate=60, enable_sps_framerate=True)

The only setting I can't (or don't know how to) set in the program is the H264 target level.

On the docs, it does mention that the H264 tops out at 1080p30, but this thread seems to suggest that is not the case.

Are there any other settings I can try? Why is the performance different between the CLI and Python?

I still need to be sampling an accelerometer while recording to check for launch/landing. Will this just make the problem worse?

Thanks in advance.

[davidplowman]

Hi, yes there are some challenges here, not least because there's quite a lot of Python code running now whereas libcamera-vid is a native C++ application. Running more Python code may cause additional problems, though course it depends what you're doing (we know that Python can't always make use of multi-threading very well, though if most of your Python tasks are sleeping or waiting for I/O then it mostly works).

Officially, only 1080p30 is supported. In practice, folks mostly manage to get up to about 1080p50 but I think anything beyond that is quite speculative.

Some other things to try:

• I assume you're running a Lite version of the OS. (Obviously I would recommend doing so if you aren't.) Also make sure you're not trying to display anything - again, I assume this is the case already.
• Set NoiseReductionMode to 3. The value 1 will still spend some time doing additional denoise which will make frames slightly late.
• In your /boot/config.txt try setting force_turbo=1. The system isn't always totally great at noticing how busy things are if you're not doing lots on the display, so pushing all the clocks up may help.
• Maybe try lowering the video bitrate. Or you might actually do better to leave the bitrate higher and reduce the resolution a bit. Or reduce the framerate a bit, it would at least be interesting to see what it will support.
• File systems can be unpredictable. Might be worth writing to /dev/null just to rule out any effects there. (If your videos are reasonably short, you might be able to write them to /dev/shm, though I realise Pi Zero 2s are a bit limited)

[Phooood]

Thanks David, sorry for the late reply. Finals just wrapped up. I am running the lite version and my only interface with it is through SSH. I have double checked to make sure force_turbo=1.

Setting NoiseReductionMode to 3 has made a big difference. I incorrectly assumed 1 meant it was off. I'm currently only dropping about 10% of frames (9s video out of a 10s recording period).

I'm not exactly sure what I should be looking for when writing to /dev/null, nor am I really sure on how to do that within the program. I'm currently writing using this line: filename = '{date:%Y-%m-%d_%H.%M.%S}.h264'.format(date = datetime.datetime.now()). I'm a bit of a noob when it comes to Linux - I have only really used it for school.

I may end up sacrificing some resolution/framerate and just be happy with the way it'll be. I'll have to play around with it a bit and see how I feel. Thanks again.

[davidplowman]

As I think I said, mostly folks find that they can't quite reach 1080p60 using Python, but they get close. The videos may be better if you reduce the framerate to (for example) 50fps, as the output can look rather choppy if it's randomly dropping frames.

To try /dev/null just use filename = '/dev/null', and see if the performance is better. This will cause the output to be discarded, so you can assess whether filesystem delays are causing frame drops. Another suggestion might be this:

filename = '/dev/shm/{date:%Y-%m-%d_%H.%M.%S}.h264'.format(date = datetime.datetime.now())

which will write to a memory filesystem. Again this should be faster than a real filesystem, but will only be possible if your files are not too large (compared to the amount of RAM your Pi has, though that could indeed be a problem on a Pi Zero 2). Also, those files get lost on a reboot, so you have to copy them to the "real" filesystem if you need to keep them.