Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Request] Add support for libvmaf_cuda filter #717

Open
HunterAP23 opened this issue Oct 26, 2023 · 3 comments
Open

[Request] Add support for libvmaf_cuda filter #717

HunterAP23 opened this issue Oct 26, 2023 · 3 comments

Comments

@HunterAP23
Copy link

HunterAP23 commented Oct 26, 2023 •
edited

The libvmaf_cuda filter is a CUDA-accelerated version of the regular libvmaf filter. Normally FFmpeg will compile it if libvmaf was built with the meson option -Denable_cuda=true, and if FFmpeg was configured with these options: --enable-nonfree --enable-ffnvcodec --enable-libvmaf
--enable-nonfree is added when the the cross_compile_ffmpeg.sh option --disable-nonfree=n is specified.

I've made a modified version of the build script to try to compile libvmaf with CUDA support, but it currently fails to compile. When setting up the environment, installing cuda, libvmaf fails to compile as it seems to not be able to find CUDA when using the option --cross-file=meson-cross.mingw.txt:

The Meson build system
Version: 1.2.3
Source dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf
Build dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build
Build type: cross build
Project name: libvmaf
Project version: 2.3.1
C compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc (gcc 10.2.0 "x86_64-w64-mingw32-gcc (GCC) 10.2.0")
C linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc ld.bfd 2.36.1
C++ compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ (gcc 10.2.0 "x86_64-w64-mingw32-g++ (GCC) 10.2.0")
C++ linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ ld.bfd 2.36.1
C compiler for the build machine: cc (gcc 7.5.0 "cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the build machine: cc ld.bfd 2.30
C++ compiler for the build machine: c++ (gcc 7.5.0 "c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the build machine: c++ ld.bfd 2.30
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: YES (/usr/bin/xxd)
Run-time dependency CUDA found: NO

src/meson.build:162:22: ERROR: Dependency lookup for cuda with method 'system' failed: Couldn't find requested CUDA module 'cudart_static'

A full log can be found at /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build/meson-logs/meson-log.txt

I crudely modified the cross_compile_ffmpeg.sh file to grab the master branch of the VMAF library and to use the -Denable_cuda=true option in the meson setup, which is where the issue mentioned above occurs. If I do not specify the --cross-file option, then the build succeeds, but obviously does not create anything usable for a Windows version. I'm not sure how to set some variable for meson to find CUDA correctly in this state.

This is the diff of my modified cross_compile_ffmpeg.sh script against the original:

1065,1067c1065
<   do_git_checkout https://github.com/Netflix/vmaf.git vmaf_git
<   export CUDA_HOME=/usr/local/cuda
<   export CUDA_PATH=/usr/local/cuda
---
>   do_git_checkout https://github.com/Netflix/vmaf.git vmaf_git v2.3.0
1074c1072
<     local meson_options="--prefix=${mingw_w64_x86_64_prefix} --libdir=${mingw_w64_x86_64_prefix}/lib -Denable_cuda=true --buildtype=release --default-library=static . build"
---
>     local meson_options="--prefix=${mingw_w64_x86_64_prefix} --libdir=${mingw_w64_x86_64_prefix}/lib --buildtype=release --default-library=static . build"
2375c2373
<     config_options="$init_options --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg  --enable-libopenh264  --enable-ffnvcodec --enable-libvmaf --enable-libsrt --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm  --enable-gnutls"
---
>     config_options="$init_options --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg  --enable-libopenh264  --enable-libvmaf --enable-libsrt --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm  --enable-gnutls"
@rdp
Copy link
Owner

rdp commented Nov 1, 2023 via email

@HunterAP23
Copy link
Author

HunterAP23 commented Nov 1, 2023 •
edited

Cross-posting this information form the vmaf repo issue here: Netflix/vmaf#1227 (comment)

After installing CUDA (I tried 10.2, 11.6, 11.8, and 12.2) and NVCC, along with all the requirements for both libvmaf and the ffmpeg builder tool, I modified the ffmpeg build script to do the following three things:

  1. Added the -Denable_cuda=true flag to the libvmaf compile options
  2. Added --enable-nonfree --enable-ffnvcodec to the ffmpeg configuration options
  3. Changed the script to pull the latest commit of the main branch rather than a specific release

After doing that, I get this issue:

The Meson build system
Version: 1.2.3
Source dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf
Build dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build
Build type: cross build
Project name: libvmaf
Project version: 2.3.1
C compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc (gcc 10.2.0 "x86_64-w64-mingw32-gcc (GCC) 10.2.0")
C linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc ld.bfd 2.36.1
C++ compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ (gcc 10.2.0 "x86_64-w64-mingw32-g++ (GCC) 10.2.0")
C++ linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ ld.bfd 2.36.1
C compiler for the build machine: cc (gcc 7.5.0 "cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the build machine: cc ld.bfd 2.30
C++ compiler for the build machine: c++ (gcc 7.5.0 "c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the build machine: c++ ld.bfd 2.30
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: YES (/usr/bin/xxd)
Run-time dependency CUDA found: NO

src/meson.build:162:22: ERROR: Dependency lookup for cuda with method 'system' failed: Couldn't find requested CUDA module 'cudart_static'

A full log can be found at /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build/meson-logs/meson-log.txt

I fixed the above issue by copying the libcudart_static.a file that gets added to my system when installing Cuda 11.6 through the cuda-toolkit-11-6 package. For me this file is in both /usr/local/cuda-11.6/targets/x86_64-linux/lib and /usr/lib/x86_64-linux-gnu, and I copied this file to the ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32/lib directory.

After doing this, the cross_compile_ffmpeg.sh script to change what normally gets written to meson-cross.mingw.txt by changing the following section:

[binaries]
c = '${cross_prefix}gcc'
cpp = '${cross_prefix}g++'
ld = '${cross_prefix}ld'
ar = '${cross_prefix}ar'
strip = '${cross_prefix}strip'
pkgconfig = '${cross_prefix}pkg-config'
nm = '${cross_prefix}nm'
windres = '${cross_prefix}windres'

and adding cuda = '/usr/bin/nvcc' to the bottom.

Afterwards attempting to run the compile script again, I'm met with a different error, this one from libvmaf itself:

[2/19] Linking target test/test_picture.exe
FAILED: test/test_picture.exe
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++  -o test/test_picture.exe test/test_picture.exe.p/test.c.obj test/test_picture.exe.p/test_picture.c.obj test/test_picture.exe.p/.._src_picture.c.obj test/test_picture.exe.p/.._src_mem.c.obj test/test_picture.exe.p/.._src_ref.c.obj test/test_picture.exe.p/.._src_thread_pool.c.obj -Wl,--allow-shlib-undefined -Wl,-O1 -pthread -Wl,--start-group -lcuda /home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libcudart_static.a -L/usr/local/cuda/lib/x64 -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status
[3/19] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
/usr/bin/nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

Not sure what the -lcuda option does specifically, but it seems to look for CUDA even though it was able to find it earlier.

@gedoensmax
Copy link

gedoensmax commented Nov 28, 2023 •
edited

@HunterAP23 -lcuda means the driver library not the cuda runtime lib. This will either require the driver library or linking against the stubs shipped with CUDA toolkit. This also has to be done when building during docker build:

$ ls /usr/local/cuda/lib64/stubs/
libcublas.so    libcurand.so      libnppial.so   libnppim.so   libnvidia-ml.so
libcublasLt.so  libcusolver.so    libnppicc.so   libnppist.so  libnvjpeg.so
libcuda.so      libcusolverMg.so  libnppidei.so  libnppisu.so  libnvrtc.so
libcufft.so     libcusparse.so    libnppif.so    libnppitc.so
libcufftw.so    libnppc.so        libnppig.so    libnpps.so

libcuda.so is what you are looking for :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rdp @HunterAP23 @gedoensmax