GraphBLAS doesn't export rmm_wrap when CUDA is enabled

[DrTimothyAldenDavis]

@mmuetzel : I tried to enable cuda in SuiteSparse/GraphBLAS by setting ENABLE_CUDA to ON in SuiteSparse/GraphBLAS/CMakeLists.txt, but the cmake reports an error about not finding rmm_wrap. I realize that you can't test the CUDA case on your end, but here's what I'm seeing. Is this something that needs to be added to GraphBLAS/rmm_wrap/CMakeLists.txt?

hypersparse $ pwd
/home/faculty/d/davis/dev2/SuiteSparse/GraphBLAS
hypersparse $ make
( cd build && cmake  .. && cmake --build . --config Release -j8 )
-- Building SuiteSparse:GraphBLAS version: v8.3.0, date: Dec 30, 2023
-- GraphBLAS C API: v2.0, date: Nov 15, 2021
-- Source:        /home/faculty/d/davis/dev2/SuiteSparse/GraphBLAS 
-- Build:         /home/faculty/d/davis/dev2/SuiteSparse/GraphBLAS/build 
-- Install lib:      lib
-- Install include:  include
-- Install bin:      bin
-- Install pkg-file: lib
-- Install rpath:    
-- Build   rpath:    /home/faculty/d/davis/dev2/SuiteSparse/GraphBLAS/build
-- Build type:    Release 
-- Fortran: /usr/bin/f95
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- Looking for CUDA
-- Find CUDA tool kit:
-- Found CUDAToolkit: /usr/local/cuda/include (found version "11.7.99") 
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- CUDA toolkit found:   TRUE
-- CUDA toolkit version: 11.7.99
-- CUDA toolkit include: /usr/local/cuda/include
-- CUDA toolkit lib dir: /usr/local/cuda/lib64
-- The CUDA compiler identification is NVIDIA 11.7.99
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CUDA: enabled
-- GraphBLAS CUDA JIT: enabled
-- GraphBLAS CPU JIT: enabled
-- GBCOMPACT: enabled; FactoryKernels will not be built
-- The CXX compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- C++ flags for CUDA:  --std=c++17 -fPIC 
-- RMM_WRAP_INCLUDES: ../rmm_wrap
-- GraphBLAS CUDA includes: ../rmm_wrap;../Source;../Source/Shared;../Source/Template;../Source/Factories;../Include;../CUDA
-- Found Python: /raid/spack/opt/spack/linux-ubuntu20.04-broadwell/gcc-9.4.0/python-3.10.12-7rih7kyn54idbhpg4ole52me22dy6nol/bin/python3.10 (found version "3.10.12") found components: Interpreter 
-- CUDA tests files: 
-- Found CUDA: /usr/local/cuda (found version "11.7") 
Cloning into 'spdlog'...
Note: switching to '76fb40d95455f249bd70824ecfcae7a8f0930fa3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Cloning into 'rmm'...
-- cpu_features (by google.com): enabled 
-- Building dynamic GraphBLAS library (no static)
-- cpu_feautures has dlfcn.h
-- cpu_feautures has getauxval from sys/auxv.h
-- ANSI C11 atomics: OK. -latomic not needed
-- CMAKE OpenMP libraries:    /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- CMAKE OpenMP include:      
-- CMAKE OpenMP C flags:      -fopenmp
-- CMAKE C flags:  -Wundef  -std=c11 -lm -Wno-pragmas  -fexcess-precision=fast  -fcx-limited-range  -fno-math-errno  -fwrapv  -O3 -DNDEBUG -fopenmp  
-- Also compiling the demos in GraphBLAS/Demo
-- ------------------------------------------------------------------------
-- JIT configuration:
-- ------------------------------------------------------------------------
-- JIT C compiler: /usr/bin/cc
-- JIT C flags:     -Wundef  -std=c11 -lm -Wno-pragmas  -fexcess-precision=fast  -fcx-limited-range  -fno-math-errno  -fwrapv  -O3 -DNDEBUG -fopenmp  -fPIC 
-- JIT link flags:  -shared 
-- JIT lib prefix: lib
-- JIT lib suffix: .so
-- JIT obj suffix: .o
-- JIT cache:      /home/faculty/d/davis/.SuiteSparse/GrB8.3.0
-- JIT openmp inc: 
-- JIT openmp dirs 
-- JIT libraries:   -lm -ldl -lgomp -lpthread
-- JIT cmake libs: m;dl;/usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- ------------------------------------------------------------------------
-- SuiteSparse CMAKE report for: graphblas
-- ------------------------------------------------------------------------
-- inside common SuiteSparse root:  false
-- install in SuiteSparse/lib and SuiteSparse/include: OFF
-- build type:           Release
-- BUILD_SHARED_LIBS:    ON
-- BUILD_STATIC_LIBS:    OFF
-- use OpenMP:           yes 
-- C compiler:           /usr/bin/cc 
-- C flags:               -Wundef  -std=c11 -lm -Wno-pragmas  -fexcess-precision=fast  -fcx-limited-range  -fno-math-errno  -fwrapv  -O3 -DNDEBUG -fopenmp 
-- C++ compiler:         /usr/bin/c++
-- C++ flags:            
-- C Flags release:      -O3 -DNDEBUG 
-- C++ Flags release:    -O3 -DNDEBUG 
-- Fortran compiler:     /usr/bin/f95 
-- compile definitions:  GBCOMPACT
-- CUDA architectures:   52;75;80
-- ------------------------------------------------------------------------
-- Configuring done (14.3s)
CMake Error: install(EXPORT "GraphBLASTargets" ...) includes target "GraphBLAS" which requires target "rmm_wrap" that is not in any export set.
CMake Error in CMakeLists.txt:
  export called with target "GraphBLAS" which requires target "rmm_wrap" that
  is not in any export set.


-- Generating done (0.2s)
CMake Generate step failed.  Build files cannot be regenerated correctly.
make: *** [Makefile:31: library] Error 1

[DrTimothyAldenDavis]

I guess it's because the rmm_wrap sub-package doesn't have the new include ( CMakePackageConfigHelpers ) and following, which all the other packages have?

[DrTimothyAldenDavis]

One option, which I'll have to bounce off of my NVIDIA collaborators, is to merge rmm_wrap into the GraphBLAS_CUDA package. That way I reduce the # of packages by one.

[mmuetzel]

I'm looking into it. Should probably be not too difficult.

[mmuetzel]

See #543.

[DrTimothyAldenDavis]

Fixed -- thanks!

[mmuetzel]

One option, which I'll have to bounce off of my NVIDIA collaborators, is to merge rmm_wrap into the GraphBLAS_CUDA package. That way I reduce the # of packages by one.

If the approach in #547 turns out to work, we could probably do the same for rmm_wrap and absorb it into GraphBLAS_CUDA.
Do we need to wait for approval from NVIDIA for that?

[DrTimothyAldenDavis]

That would be a good idea; it would simplify the linking of the packages for the end user application.

I'll double-check with NVIDIA. Since the CUDA acceleration for GraphBLAS isn't ready for production anyway, let's wait until after I post a stable SuiteSparse 7.4.0, with rmm_wrap as-is.