Migrating from nvbuf_utils to NvUtils. #125

douo · 2023-02-23T16:51:26Z

Refactor the code to migrate from nvbuf_utils to NvUtils, and it also includes fixes for issues #120 #123 and #115 . The changes have been tested in Jetpack 5.1.

douo · 2023-02-23T16:52:33Z

Ref: https://developer.nvidia.com/sites/default/files/akamai/embedded/nvbuf_utils_to_nvutils_migration_guide.pdf

douo · 2023-02-24T01:14:38Z

It's not backwards compatible. I will keep the branch separate for Jetpack 5.x only.

douo · 2023-02-24T14:31:28Z

I discovered some performance issue on jetpack 5.1.

devices(MAXN jetson_clocks):

AGX Jetpack 5.1 L4T 35.2.1
Nano Jetpack 4.6.3 L4T 32.7.3

Command:

nvv4l2dec: ffmpeg -y -benchmark -c:v hevc_nvv4l2dec -i $input -f null -
gstreamer: gst-launch-1.0 filesrc location=$input ! h265parse ! nvv4l2decoder enable-max-performance=1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
00_video_decode: video_decode H265 --disable-rendering --stats $input
jetson_ffmpeg: ffmpeg -y -benchmark -c:v hevc_nvmpi -i $input -f null -

sample_3840x2160.hevc:

Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 23.98 fps, 23.98 tbr, 1200k tbn, 23.98 tbc

	AGX(fps)	Nano(fps)
00_video_decode	32.7	97.96
gstreamer	232.61	97.71
nvv4l2dec	24	63
nv_mpi	28	70
ffmpeg cpu	25	7.3

sample_4k.h264:

Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 3840x2160, 25 fps, 25 tbr, 1200k tbn, 50 tbc

	AGX(fps)	Nano(fps)
00_video_decode	25.7	88.10
gstreamer	132.84	87.29
nvv4l2dec	25	63
nv_mpi	23	69
ffmpeg cpu	64	19

sample_720.h264:

Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1280x544, 24.08 fps, 23.98 tbr, 1200k tbn, 47.95 tb

	AXG(fps)	Nano(fps)
00_video_decode	71.18	770.58
gstreamer	733.70	764.39
nvv4l2dec	47	409
nv_mpi	46	490
ffmpeg cpu	676	270

bmegli · 2023-03-07T14:09:45Z

@douo

Could you try adding setMaxPerfMode

	ret=ctx->dec->setMaxPerfMode(1);
	TEST_ERROR(ret < 0, "Error in setting decoder maximum performance mode", ret);

Right after lines in your code

	ret=ctx->dec->setCapturePlaneFormat(format.fmt.pix_mp.pixelformat,format.fmt.pix_mp.width,format.fmt.pix_mp.height);
	TEST_ERROR(ret < 0, "Error in setting decoder capture plane format", ret);

Context

I am using AGX Orin. Setting performance mode brings decoding roughly to GStreamer level.

dpkg-query --show nvidia-l4t-core
nvidia-l4t-core	34.1.1-20220516211757

apt-cache show nvidia-jetpack
Package: nvidia-jetpack
Version: 5.0.1-b118

Test

GStreamer

input=~/Downloads/iphone6s_4k.mov

gst-launch-1.0 filesrc location=$input ! qtdemux ! h264parse ! nvv4l2decoder enable-max-performance=1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 542, dropped: 0, current: 119,99, average: 119,32

your fork

./ffmpeg -y -benchmark -c:v h264_nvmpi -i ~/Downloads/iphone6s_4k.mov -f null -

frame=  549 fps= 78 q=-0.0 Lsize=N/A time=00:00:18.55 bitrate=N/A speed=2.65x

your fork + change mentioned by me

./ffmpeg$ ./ffmpeg -y -benchmark -c:v h264_nvmpi -i ~/Downloads/iphone6s_4k.mov -f null -

frame=  551 fps=111 q=-0.0 Lsize=N/A time=00:00:18.55 bitrate=N/A speed=3.75x

bmegli · 2023-03-07T14:13:18Z

Here is also some inspection what GStreamer nvvideo4linux2 plugin does when setting performance mode:

encoder maxperf-enable path Extend-Robotics/gst-nvvideo4linux2#1

I did it for encoder, but decoder ends up in same function call.

douo · 2023-03-08T02:54:24Z

Great. The fps has significantly improved, but there is still a considerable gap compared to gstreamer in AGX Xavier.

4kh265: 28 -> 64
4kh264: 23 ->57
720ph264: 46 -> 290

…r#123 jocover#115

bmegli · 2023-03-16T15:01:39Z

@douo

After flashing AGX Jetpack 5.1 L4T 35.2.1 (on AGX Orin 32 GB)

cat /etc/nv_tegra_release 
# R35 (release), REVISION: 2.1, GCID: 32413640, BOARD: t186ref, EABI: aarch64, DATE: Tue Jan 24 23:38:33 UTC 2023

apt-cache show nvidia-l4t-core

Package: nvidia-l4t-core
Version: 35.2.1-20230124153320

I can confirm performance loss in H.264 decoding

./ffmpeg -y -benchmark -c:v h264_nvmpi -i ~/Downloads/iphone6s_4k.mov -f null -

frame=  551 fps= 39 q=-0.0 Lsize=N/A time=00:00:18.55 bitrate=N/A speed=1.32x

Citing my result with 34.1.1 (jetpack 5.0.1)

./ffmpeg -y -benchmark -c:v h264_nvmpi -i ~/Downloads/iphone6s_4k.mov -f null -

frame=  551 fps=111 q=-0.0 Lsize=N/A time=00:00:18.55 bitrate=N/A speed=3.75x

So dropped from 111 to 39 fps (both with max-perf-enable)

Keylost · 2023-03-20T19:33:00Z

U can try my repo
https://github.com/Keylost/jetson-ffmpeg
AGX Xavier JP5.1 (Power Mode: MAXN) H264 3840x2160 decoding: ~140 fps
I will probably improve it a little more in the future.
p.s. I haven't added setMaxPerfMode yet. I'm not sure if it's right to forcefully set this mode...

bmegli · 2023-03-21T08:02:45Z

U can try my repo
https://github.com/Keylost/jetson-ffmpeg
AGX Xavier JP5.1 (Power Mode: MAXN) H264 3840x2160 decoding: ~140 fps

Awesome!

./ffmpeg -y -benchmark -c:v h264_nvmpi -i ~/Downloads/iphone6s_4k.mov -f null -
# ...
frame=  551 fps=115 q=-0.0 Lsize=N/A time=00:00:18.55 bitrate=N/A speed=3.88x

Quick test confirms performance restored on AGX Orin

111 with @douo variant + max-perf-enable in 34.1.1
39 with @douo variant + max-perf-enable in 5.1 / 35.2.1
115 with @Keylost variant without max-perf-enable in 5.1 / 35.2.1
120 with gstreamer, max-perf-enable doesn't affect result in 5.1 / 35.2.1

I haven't checked correctness of decoded video

p.s. I haven't added setMaxPerfMode yet. I'm not sure if it's right to forcefully set this mode...

I guess it's more about

baseline as gstreamer with nvv4l2decoder enable-max-performance=1
performance loss of the same code in 5.1, including Nvidia examples
Xavier/Orin performing below Nano level

But it looks like you pinpointed the performance loss cause.

Keylost · 2023-03-21T11:25:54Z

I'm glad to hear that the decoder performance restored on AGX Orin as well :)

Please note: in order to use the variant from my repository, you need to build ffmpeg with a patch from my repository.

eusoubrasileiro · 2023-07-24T17:43:57Z

@Keylost and @douo and @bmegli Thanks a lot! And congratulations for you all on your continuous effort to improve this repository that seams abandoned!

@Keylost

jetson-ffmpeg to ffmpeg version 6.0 Upgrade motion to version 4.5 Replace jetson-ffmpeg by @Keylost repository. Altough, @douo and @bmegli have a good too look at. Here jocover/jetson-ffmpeg#125 references.

geoncoder · 2023-10-22T16:22:06Z

This PR migrates the decoder to NvUtils. Is there another PR which also migrates the encoder? I get an error during cmake trying to find nvbuf_utils in the encoder.

Migrating from nvbuf_utils to NvUtils.

d985684

douo closed this Feb 24, 2023

douo reopened this Feb 24, 2023

bmegli mentioned this pull request Mar 7, 2023

open nvmpi decoder fails at Nvidia ORIN plateform #120

Open

Sets the decoder for maximum performance.

e96d1ac

Keylost added a commit to Keylost/jetson-ffmpeg that referenced this pull request Mar 14, 2023

added nvutils api support. nvbuf_utils compatible. jocover#125 jocove…

db39c6b

…r#123 jocover#115

This was referenced Mar 15, 2023

clarify Nvidia and ATI/AMD situation bmegli/hardware-video-encoder#15

Closed

L4T 5.1 missing return statement in encoder causing segmentation fault #127

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrating from nvbuf_utils to NvUtils. #125

Migrating from nvbuf_utils to NvUtils. #125

douo commented Feb 23, 2023 •

edited

douo commented Feb 23, 2023

douo commented Feb 24, 2023

douo commented Feb 24, 2023

bmegli commented Mar 7, 2023

bmegli commented Mar 7, 2023

douo commented Mar 8, 2023

bmegli commented Mar 16, 2023 •

edited

Keylost commented Mar 20, 2023

bmegli commented Mar 21, 2023 •

edited

Keylost commented Mar 21, 2023 •

edited

eusoubrasileiro commented Jul 24, 2023

geoncoder commented Oct 22, 2023

Migrating from nvbuf_utils to NvUtils. #125

Are you sure you want to change the base?

Migrating from nvbuf_utils to NvUtils. #125

Conversation

douo commented Feb 23, 2023 • edited

douo commented Feb 23, 2023

douo commented Feb 24, 2023

douo commented Feb 24, 2023

bmegli commented Mar 7, 2023

Context

Test

GStreamer

your fork

your fork + change mentioned by me

bmegli commented Mar 7, 2023

douo commented Mar 8, 2023

bmegli commented Mar 16, 2023 • edited

Keylost commented Mar 20, 2023

bmegli commented Mar 21, 2023 • edited

Awesome!

Keylost commented Mar 21, 2023 • edited

eusoubrasileiro commented Jul 24, 2023

geoncoder commented Oct 22, 2023

douo commented Feb 23, 2023 •

edited

bmegli commented Mar 16, 2023 •

edited

bmegli commented Mar 21, 2023 •

edited

Keylost commented Mar 21, 2023 •

edited