entry point not found

[tianweimol]

Description

my eveniroment:
RTX4060TI
VS2022
.net8 or .net6
cuda:11.2
cudnn:8.1.1.33

nuget install
TensorFlow.Net 0.150.0
TensorFlow.Keras 0.15.0
SciSharp.TensorFlow.Redist-Windows-GPU 2.10.3

i have clone the project:SciSharp-Stack-Examples
run TensorFlowNET.Examples/GAN/MnistGAN.cs
the function named：predictImage
the code below will throw a Exception:

var tensor_result = g.predict(noise);

exception is :

System.EntryPointNotFoundException:“Unable to find an entry point named 'TF_GetHandleShapeAndType' in DLL 'tensorflow'.”

what happend?
i guess the reason is : version of cuda or cudnn is wrong.but i don't know which version i should use.

thanks for you answer

[PavelBakurov]

Same here. Installed CUDA 11.7.1

[tianweimol]

i have install cuda 11.7.1,cudnn 8.1 for 11.x.
when the code bellow is running

var tensor_result = g.predict(noise);

i still get the excption:

System.EntryPointNotFoundException:“Unable to find an entry point named 'TF_GetHandleShapeAndType' in DLL 'tensorflow'.”

i have try the below version of cudnn for cuda11.7

8.9.6
8.9.5
8.9.4
8.9.3
8.9.2
8.9.0
8.8.1
8.8.0
8.7.0
8.6.0
8.5.0
8.4.1
8.4.0

and the code still exception.
which version of cuda and cudnn should i choose?
thanks for you answer.

execute:nvidia-smi
output:
Mon Jan  8 10:28:57 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 546.33                 Driver Version: 546.33       CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4060 Ti   WDDM  | 00000000:1B:00.0 Off |                  N/A |
|  0%   25C    P8               6W / 165W |   1050MiB / 16380MiB |      0%      Default |
|                                         |                      |                  N/A

execute:nvcc -V
output:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:59:34_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0

[Aglathal]

I had the same issue (with the same versions).
Solution was to downgrade packages to the following versions:

<PackageReference Include="TensorFlow.Keras" Version="0.10.5" />
<PackageReference Include="TensorFlow.NET" Version="0.100.5" />

Now it works on GPU... but interesting thing is it is actually slower than on CPU - so I am still doing something wrong.

[Aglathal]

Ehh I give up - I hoped to switch from python to c# but the same model and training data (mnist) works ~8x slower in TensorFlow.NET (both frameworks using CPU). And GPU makes no difference (at least in case of TensorFlow.NET).
Could be useful for inference later but definitely not for training.

[mvphelps]

Thanks @Aglathal for your comment RE package references, this got mine working. The GPU was only about 30% faster than CPU, but it was faster (on a wimpy 1070 GTX). My custom model that is nothing like MNist has gone from training in 11 hours to about 20 minutes.

For any future people, this configuration utilizes GPU as of 2024 FEB 26:

Dotnet 8
CUDA 11.8
CUDNN v8.6.0.163
TensorFlow.Keras 0.10.5
TensorFlow.NET 0.100.5
SciSharp.Tensorflow.Redist-Windows-GPU 2.10.3
Do NOT have SciSharp.Tensorflow.Redist installed.

Also note if you are doing online prediction, there is a GC.Collect call inside the predict method, and that really wrecks performance. To work around that, use a little reflection and cache a MethodInfo to reuse on each prediction, like this:

private void GetPredictMethod()
{
    //Signature of method we are targeting: Tensors predict_step(Tensors data)
    var type = typeof(Model);
    //Cache this in a field, and invoke it when predicting instead of the public .predict which calls GC.
    _predictMethod = type.GetMethod("predict_step",
        BindingFlags.InvokeMethod | BindingFlags.NonPublic | BindingFlags.Instance);
}