ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

[glenbhermon]

My cuda toolkit version is 12.1 & nvcc -V show the following results
vcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Feb__7_19:32:13_PST_2023 Cuda compilation tools, release 12.1, V12.1.66 Build cuda_12.1.r12.1/compiler.32415258_0
when i ran for checking cudnn version by the following command :

cat /usr/include/x86_64-linux-gnu/cudnn_v*.h | grep CUDNN_MAJOR -A 2

then it show the following things:

#define CUDNN_MAJOR 9
#define CUDNN_MINOR 0
#define CUDNN_PATCHLEVEL 0

#define CUDNN_VERSION (CUDNN_MAJOR * 10000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)

/* cannot use constexpr here since this is a C-only file */

---

In .bashrc these things are added

export PATH=/usr/local/cuda-12.1/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} export PATH=$PATH:~/.local/bin

After following the instruction and running the command from README_LINUX.md
GPLOK=1 bash docs/linux_install.sh

It install the necessary libraries but when I import the pytorch then it throw the following error:

ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

[pseudotensor]

Did you do this step mentioned in the readme_linux.md?

echo 'export CUDA_HOME=/usr/local/cuda-12.1' >> $HOME/.bashrc
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_HOME/lib64:$CUDA_HOME/extras/CUPTI/lib64' >> $HOME/.bashrc
echo 'export PATH=$PATH:$CUDA_HOME/bin' >> $HOME/.bashrc

and restart the shell?

[glenbhermon]

yes, i first do the steps that are mentioned in readme_linux.md and rebbot but it was not worked

[pseudotensor]

Can you (with that env) even import torch and check if cuda is avail like in readme? i.e.

import torch
print(torch.cuda.is_available())

[glenbhermon]

yes, it throws the "ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory"

[pseudotensor]

Ok, then your problem is very fundamental and not related to h2oGPT. You should try to create an environment (virtualenv or conda), pip install torch, and be able to run that check.

[glenbhermon]

I think the problem is related to the pytorch compatibility with cudnn 9.0.0. h2ogpt works perfectly with cuda toolkit 11.8 and cudnn 8.7.x