Well that's unfortunate. We'll have to do version checking to switch between the two pixel format names, and parsing of ffmpeg's version string has been difficult in the past.

How about checking available pix_fmts and picking one? Wouldn't that be more universal? I would assume parsing that would be even harder

• Test out of the box functionality with ffmpeg 5 (using x264)
• Test nvenc
• Test omx (Raspberry Pi)
• Find a solution to support vaapi_vld and vaapi pixel formats to support both ffmpeg 4 and ffmpeg 5.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this was a feature request that others have shown no interest in then it's likely to not get implemented due to lack of interest. If others also want to see this feature then now is the time to say something! Thank you for your contributions.

Due to the limited number of people using dedicated hardware this hasn't been more of an issue, but it's still on the todo list.

An update:

As far as I can tell ffmpeg 5 has been working for everyone except those trying to use vaapi, and I've only been made aware of two people trying to use ffmpeg5+vaapi. Due to the lack of impact, this has not been a priority.

However, if anybody else wanted to pick this up to test, verify and support the new pixel format it'd be great. Otherwise it'll continue to be pushed off until the larger projects that are in the works are released.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this was a feature request that others have shown no interest in then it's likely to not get implemented due to lack of interest. If others also want to see this feature then now is the time to say something! Thank you for your contributions.

Just wanted to add that upgrading to ffmpeg 6 if you use x264 shouldn't be a problem.

I have been running it for a week 24/7 now. The performance of using x264 with ffmpeg 6 on an ARM CPU was also noticeably better than with ffmpeg 4 or ffmpeg 5. I'm seeing about 13% lower CPU with the same settings (three variants on highest quality).

I can take a look at this if the fix isn't required too soon. Being able to detect the ffmpeg capabilities is probably also needed for the other hardware encoding improvements I want to do.

No rush, it's been sitting around as a todo this long :)

Maybe we're overthinking it. Maybe we treat these codec implementations as completely different codecs? Like we could have Vaapi Original and Vaapi New or something and just have the user-facing display name be "Vaapi" for both.

For a short term solution it would probably be the easiest. But I'm a bit worried about the possible number of options this would create in the long term. At least for NVENC I could imagine around 3 different options it could run with (different hardware scalers).
Unless it would try to run each codec at the start to filter out the incompatible ones.

I didn't realize there was an issue with nvenc as well, I thought the only issue we were trying to resolve was the renaming of vaapi with the new versions of ffmpeg. What is the nvenc issue with ffmpeg >5?

Unless it would try to run each codec at the start to filter out the incompatible ones.

I don't think that's going to be possible since failures can happen for so many different reasons.

Ah sorry for being unclear. This is more of an issue with the general hardware acceleration for nvenc and has nothing to do with the new ffmpeg version. It's just that there are two different ways to enable hardware accelerated rescaling with nvenc which might not always be present. In that case you would need to provide different options to the user.

Oh got it, thanks for clarifying. If it's just two versions of nvenc then we could write code to introspect the options available to nvenc and try to parse out the version, kind of like we're doing with codec names, but with codec options instead. But if it's more than two versions of a codec we need to support than I don't think that's a scalable solution.

Hello, what do you think about #3071 ?

Hi, I'm currently running owncast on an LXC Container using Alpine base os, and I'm currently facing the same issue.
The FFMPEG version on Alpine 3.18 (current latest) is >6.0.1, and beside the hevc hw acceleration, even the x264 hw support breaks owncast.
As far as I test, the last ffmpeg version (v6) leads to a very big performance improvement compared to the v4 suggested for owncast. On a lower specs single-board pc like a Raspberry Pi 3, there was a huge improvement in term of CPU usage, memory and sw transcoded segments size/quantity leading to a faster response from the application (tried with jellyfin).
Now I'm using an AMD CPU with integrated graphics (IG), using latest Proxmox and set IG passthrough into owncast LXC on Alpine successfully. The usage of the hw acceleration using ffmpeg >5 (or v6 better) even for just h264, would give a huge help to the CPU usage.
I don't know if someone is going to work on this issue, unfortunatelly I'm not that skilled to help in coding, but I can help in testing if needed.
Thank you for your work guys

it just generates the hls files and segments and complains about low memory.

After I removed the presets I also get low memory warnings, however, it's working fine without issues. I'd like to resolve those warnings, but at least it's working.

@mahmed2000 Since you're testing manually, could you share the invocation you're testing with? I'll run the same thing and report back.

Hi,
I'm trying this on Alpine 3.19, I never did that before, so I'm going to explain what I did:

• made a git clone of owncast selecting gek/ffmpeg-6 as branch
• installed os needed package: ffmpeg(6.1.1-r0) go(1.21.9-r0) musl(1.2.4_git20230717-r4) git nodejs(20.12.1-r0) icu-data-full npm python3(3.11.8-r0) mesa-va-gallium(23.3.6-r0)
  To let backend compiling successfully, I had to edit a required go package: go-sqlite3 from v1.14.17 to v1.14.22, but I already checked that the last owncast version already had this dependency bump done.
  
  As you can see from the screen above, the new option is there.

After enable it, I got an error on backend console but the streaming seems working fine for now.

Please let me know if I did something wrong during my steps or if you need some data or logs that I'm not aware of.

Additionally

The argument is a small integer, from 1 up to some limit dependent on the device (not more than 7) - higher values are faster / lower stream quality.

But it also says that 0 is speed preset only and 29 is quality. I might be misreading what this example means. I initially thought 0 meant the fastest, but now I think 0 might be the slowest. It also says that 0 isn't even a valid option, but is shown in the example.

@gabek Yep, you're right with the compression_level setting. Varying the presets doesn't seem to have any effect on file size, but compression_level is having an effect on Intel (only 1 and 2 seem to actually change anything though). Still, default behavior seems to be to ignore rather than throw an error. Docs just kinda seem all over the place for this.

Command, testing with OBS, just modified file outputs, debug level logs and listen for RTMP, redirect stderr to file:

/usr/bin/ffmpeg -hide_banner -loglevel debug -hwaccel vaapi -hwaccel_output_format vaapi -vaapi_device /dev/dri/renderD128 -fflags +genpts -flags +cgop -listen 1 -f flv -i rtmp://127.0.0.1:1935 -map v:0 -c:v:0 h264_vaapi -b:v:0 1008k -maxrate:v:0 1088k -g:v:0 75 -keyint_min:v:0 75 -r:v:0 25 -map "a:0?" -c:a:0 copy -sws_flags bilinear -filter:v:0 "scale_vaapi=1280:720" -compression_level -1 -map v:0 -c:v:1 h264_vaapi -b:v:1 2808k -maxrate:v:1 3032k -g:v:1 75 -keyint_min:v:1 75 -r:v:1 25 -map "a:0?" -c:a:1 copy -compression_level -1 -var_stream_map "v:0,a:0 v:1,a:1 " -f hls -hls_time 3 -hls_list_size 10 -hls_flags program_date_time+independent_segments+omit_endlist -segment_format_options "mpegts_flags=mpegts_copyts=1" -pix_fmt vaapi -sc_threshold 0 -master_pl_name stream.m3u8 -hls_segment_filename "%v/stream-cwDO2lxIg-%d.ts" -max_muxing_queue_size 400 "%v/stream.m3u8" 2> t.log

@LelieL91 Thanks so much for testing! The more datapoints the better! Continue to watch this thread if you can, I'm sure we'll be making changes and will continue to be testing.

We do have a way to ignore errors that are non-fatal and might be due to buggy codec log messages, or other things out of our control. Not something we should just throw things into right away, but just mentioning that there's a precident of dealing with "we're getting errors, but it works fine" scenarios.

https://github.com/owncast/owncast/blob/develop/core/transcoder/utils.go#L45

GitHub

owncast/core/transcoder/utils.go at develop · owncast/owncast

Take control over your live stream video by running it yourself. Streaming + chat out of the box. - owncast/owncast

@LelieL91 Thanks so much for testing! The more datapoints the better! Continue to watch this thread if you can, I'm sure we'll be making changes and will continue to be testing.

Sure, no problem, if you remember I already said that I'm happy to help about this feature.
My tests are made on LXC containers (Proxmox VE), CPU: Ryzen 7 5800U iGPU: Radeon Vega 8.
I tried running owncast before the GPU passthrough and I got an error about missing access to the gpu /dev/dri/renderD128.
So I think the GPU passthrough on a LXC unprivileged container is possible, Alpine seems having all gpu drivers (intel/amd), I can provide the list of packages available if needed. At the moment, the mesa-va-gallium seems enough.

ffmpeg -hwaccels
ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
  configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   5.  0.100 /  5.  0.100
  libpostproc    57.  3.100 / 57.  3.100
Hardware acceleration methods:
vdpau
vaapi
qsv
drm
vulkan

If you need something specific, feels free to ask anytime. At the moment, this error is the only "issue" but not a fatal one.

I don't know if possible, but upgrading this fork with the current dependencies/commits would be helpful for future test/merging, the problem with the go module was kind annoying to find out. Luckily it was only that one for now 🥲
But always thank you for your time and work for all of us

I maybe found something online about my error, I'm going to report here, could be maybe useful for someone else:
ERROR: [h264_vaapi @ 0x7dacab62c780] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).

• Intel CPU are not affected by this issue, only AMD CPU.
• It seems that the software use cpu has fallback, so the hw support on amd seems kind not working using the default mesa driver
• Jellyfin seems applying the same "fix" at the end of the url below, but that would require a custom ffmpeg package

Look at the link for referencies: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3524#note_1989027

GitLab

error message when encoding via VAAPI AMD (#3524) · Issues · Mesa / mesa · GitLab

Do ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -ss 15 -i input.mp4 -t 15 -c:v h264_vaapi -global_quality 15 -c:a copy -map 0 output.mp4 Then this...

@LelieL91 that's definitely a driver issue, what's the output of vainfo?

Also as per the Alpine wiki did you install mesa-dri-gallium?

Additionally

The argument is a small integer, from 1 up to some limit dependent on the device (not more than 7) - higher values are faster / lower stream quality.

But it also says that 0 is speed preset only and 29 is quality. I might be misreading what this example means. I initially thought 0 meant the fastest, but now I think 0 might be the slowest. It also says that 0 isn't even a valid option, but is shown in the example.

@gabek I think its dependent on AMD v other devices. AMD follows the format in its section with 0-29 which is more of a sequence of bit flags than an actual scale. For other encoders it maps to some arbitrary scale dependent on the device. Could be 1-7, could be 1-5, could be 1-3. Might map directly onto the presets of say h264_qsv for example which also go from 1-7 or does some custom settings. That does complicate things.

I was also testing it wrong and compression_level actually does seem to change speed upto 7 on my end.

There still has to be an answer, though. We can't start asking people what their hardware is. Maybe the real solution is to not provide a level at all and hope whatever the defaults are will be good enough. Perhaps not the most optimal, but if there isn't an answer to this question, then there's not much that can be done.

Did some digging into compression_level, from the ffmpeg source, the only reference in the context of h264_vaapi (outside of setting the default) is in this function

In short, it checks it against the VAAPI interface for this VAConfigAttribEncQualityRange. The value gets set by vaapi itself, and the actual range would depend on the underlying device and driver, which is why the ffmpeg docs say what they say. What the value means is also up to the device as with the AMD section.

I should be seeing a warning when setting the value arbitrarily high (1000, etc) but it seems to just be implicitly clamping down to 1-7 somewhere else. Does a value >30 trigger a warning Invalid quality level: valid range is 0-%d, using %d with AMD?

We can't start asking people what their hardware is.

vainfo is an option, but its an extra dependency to libva-utils. Should still do something about this for multi-device cases where the user might want to specify which, or if renderD128 maps to the iGPU instead of the desired dGPU. Apparently the order may also not be consistent across boots.

hope whatever the defaults are will be good enough

That can work, but then the CPU or GPU usage slider is misleading for vaapi. Could go the obs route and require the user to set the flag and value explicitly. See obsproject/obs-studio#7953 (comment).

vainfo is an option, but its an extra dependency to libva-utils.

Unfortunately, I think that's a non starter. I know most people will have it installed already to get vaapi working in the first place, but that's too much of a dependency.

Could go the obs route and require the user to set the flag and value explicitly.

That sounds like a support nightmare. It would be the most customizable, but in reality nobody is going to understand how to set this value, and it will become my problem. This is the same reason you can't just type in a free form framerate in the admin, for example.

the CPU or GPU usage slider is misleading for vaapi

That's the core of it all for me. Is there a way to give people a realtive way to approximately provide their intention. I understand a 3 on Intel is different than a 3 on AMD. But is a 3 on Intel faster than a 0 on Intel, and is a 3 on AMD faster than a 3 on AMD? If so, then we can at least give some control, and that's better than none.

Is there a way to give people a realtive way to approximately provide their intention.

If it was AMD only, yes. If it was Intel only, yes. Together, no. A 3 on Intel is faster than a 0. A 3 on AMD is slower than a 0, but faster than a 1, and 2 isn't even valid. Either is incompatible with how the other uses the value and needs handling separately, which is why I linked the OBS comment. Also need to figure out where NVIDIA fits.

Intel

I can time the ffmpeg call by running it on a specific file ~1MB. It's a gradual decrease per level from ~2.15s at compression_level 1, over to ~1.45s at compression_level 7, within ~0.05s of variation. All values in the range are valid. It should clamp to the max value supported by the device which may be lower, but its faster as the number goes up. Basically:

• 0: slowest
• .
• .
• .
• 7: fastest

AMD

Take the following with a heavy dose of testing, I don't have AMD hardware to verify on. Just what I understand from the docs.
AMD does it in reverse. For some reason VBAQ and pre-encode settings have also been added in with this value as higher order bits. Pre-encode has a performance hit, don't know how bad exactly. Additionally, only odd numbers should be valid past 0 due to the "validity bit". Ignoring those settings, we have left:

• 0: speed, fastest
• 1: forces balanced with VBAQ and pre-encode, slower than both 0 and 3, could be skipped as stated above.
• 3: balanced basic, mid
• 5: quality basic, slowest
  This is a clean-ish mapping but will be a problem if someone wants to enable pre-encode or VBAQ. Again, incompatible with Intel, but consistent across AMD.

but that's too much of a dependency.

Would just libva with some C code work? Just outputting the driver version string from vainfo is fairly trivial if it's enough.

Otherwise, it's just not possible as is.

Would just libva with some C code work? Just outputting the driver version string from vainfo is fairly trivial if it's enough.

It could be a decent solution. I'm wondering if it's worth adding keeping track of hardware, since that's not currently a thing we do. I don't know how many people actually take advantage of vaapi. I don't think it's many. Otherwise, I'm with you, it sounds like the default will have to do and disable the slider.

Thanks for all the detailed research into this.

I want to enumerate what I'm aware of being the open questions here:

1. @rmens Was not able to set a custom resolution. I was able to.
2. @rmens Was able to run with passing presets to the codec. I was not able to.
3. The cpu/gpu usage slider needs to be disabled for vaapi.
4. @LelieL91 received "Driver does not support some wanted packed headers". I'm not sure if this is a fatal message or not.
5. Error while filtering: Cannot allocate memory Support ffmpeg 6 #2071 (comment)

Am I missing anything?

And also the Error while filtering: Cannot allocate memory. This should be ignorable, and not something that could be fixed through owncast anyway. It's specific to the hardware scaling filter. In debug level logs it only triggers after the EOF and decoder terminated messages. Might lose the last few frames when the stream ends but shouldn't affect anything else.

This triggers it:

ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_output_format vaapi -vaapi_device /dev/dri/renderD128 -i ~/Videos/test.flv -filter:v "scale_vaapi=1280:720" -c:v h264_vaapi -c:a copy stream.m3u8

But this doesn't:

ffmpeg -loglevel warning -hwaccel vaapi -hwaccel_output_format vaapi -vaapi_device /dev/dri/renderD128 -i ~/Videos/test.flv -c:v h264_vaapi -c:a copy stream.m3u8

I can also confirm even running through owncast itself, there is no error when running with presets.

I'm seeing [vf#0:0 @ 0x555dfa717840] Task finished with error code: -12 (Cannot allocate memory) as a fatal error now with a build of ffmpeg I just made from source. (N-114831-g3cca8dfbd8 --enable-gpl --enable-libx264 --enable-vaapi --enable-nonfree)

So, unfortunately, there appears to be some case(s) where it's not ignorable.

Edit: Intel put in a patch a few days ago for this? https://patchwork.ffmpeg.org/project/ffmpeg/patch/20240410030103.520402-2-haihao.xiang@intel.com/

Edit 2: I applied that patch and the memory error completely went away.

This isn't helpful now, but I did put in a pre-order of an AMD Framework laptop last November, that I'm guessing I should get around June. So at that point I should be able to test AMD + vaapi. But that's a long time to wait.

@gabek What I did to test Intel is grabbing a server from the Hetzner Server auction. They charge per hour for bare metal hardware and with the auction servers there is no set-up fee. Grab an AMD that has an integrated GPU and for pennies per hour it's testable.

That's a great suggestion, thank you!

@rmens, do you have the exact CPU and/or distro you used? Tried an i7-6700, doesn't seem to have the problem with scaling.

Unfortunately, all the AMD CPUs are all X or P series which don't have iGPUs, neither do the 3900s.

@mahmed2000 I tried an i5-12500 with Debian 12

@rmens can't repro. Still works on an i5-12500 with deb 12. I've tried the stock bookworm v5 ffmpeg, the latest autobuild from this repo: https://github.com/BtbN/FFmpeg-Builds, and manually compiling v6.1.1. I'd try your linked ppa but I can't get it to install.

Can you test and see if either the standalone bin or manually compiling sill causes the error?

GitHub

GitHub - BtbN/FFmpeg-Builds

Contribute to BtbN/FFmpeg-Builds development by creating an account on GitHub.

Bumped up from bookworm to trixie since that has 6.1.1, works fine there too.

I'll have an AMD laptop in my hands within the next couple of weeks, so I'll be able to test with those integrated graphics.

Good news. I'm testing on the integrated graphics on my new AMD laptop and it works fine. It's throwing the [h264_vaapi @ 0x55d2b5d267c0] Driver does not support some wanted packed headers (wanted 0xd, found 0x1). that we've seen, but isn't impacting anything.

Good news. I'm testing on the integrated graphics on my new AMD laptop and it works fine. It's throwing the [h264_vaapi @ 0x55d2b5d267c0] Driver does not support some wanted packed headers (wanted 0xd, found 0x1). that we've seen, but isn't impacting anything.

As I wrote before, the problem is that there is a fallback to the cpu, so yes, apparently everything seems working, but the CPU usage is the same as without gpu passthrough. And that's not only my test, as it was also mentioned in the reference above.
The only working workaround I know is what it was done on Jellyfin, but that require a custom ffmpeg package.

Can you please check your CPU usage in both with and without iGPU use?
Unfortunatelly I had some health issue, so I couldn't test more or make screenshots about that.

Sorry for the confusion. It does look like the CPU usage is the same between the two. I wonder how that even works, does vaapi have CPU encoding support? Does it import libx264? I wouldn't think it would because why? But that's beside the point.

If there's legitimately an issue with vaapi support, then I think we're stuck. Even if we wanted to ship a version of ffmpeg with Owncast, it would have to support all the codecs, including NVENC, and that's non-open software, so legally I don't believe it would be possible. Not that I'm interested in going that route anyway. We're too small of a project to be maintaining our own build of ffmpeg.