DLL load failed for Tensorflow-GPU==1.14, with CUDA 10.0

[tueboesen]

System information
CUDA Version: 10.0
CUDNN Version: 7
OS: Windows 10
Python version: 3.6
GPU: Geforce mx150

I cloned a github repository that requires tensorflow 1.13-1.14, so I just made a new virtual environment and installed tensorflow-gpu==1.14, which installed without any problems.

Then I tried to run a session in order to see that things were working:

import tensorflow as tf
sess = tf.Session()

However, when I run the above, I get the following error:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\alyfl\.virtualenvs\trRosetta_gpu\lib\site-packages\tensorflow\__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\alyfl\.virtualenvs\trRosetta_gpu\lib\site-packages\tensorflow\python\__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\alyfl\.virtualenvs\trRosetta_gpu\lib\site-packages\tensorflow\core\framework\graph_pb2.py", line 7, in <module>
    from google.protobuf import descriptor as _descriptor
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\alyfl\.virtualenvs\trRosetta_gpu\lib\site-packages\google\protobuf\descriptor.py", line 48, in <module>
    from google.protobuf.pyext import _message
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified procedure could not be found.

I just installed CUDA 10.0, since I realized that this old version of tensorflow would require that version (and restarted afterwards). I have confirmed that CUDA 10.0 is in my path environment, and that it seems to be the version linked in python as shown below:

from tensorflow.python.platform import build_info as tf_build_info
print(tf_build_info.cuda_version_number)
>10.0
print(tf_build_info.cudnn_version_number)
>7

I'm not really sure what exactly is going wrong here, and the error messages aren't much help either at this point

[Saduf2019]

@tueboesen
Can you please refer to this comment for the cuda version and compatibility.

Please, check Your CPU/Python is on 32 bits?Please, refer #36167 and see if it helps you.
similar issues #21719 #36167 #36151 #36138 #36054 #36045 #36020 #36003 #35988 #35903 #35880 #35865 #35805 #35789 #35773 #35772 #35767 #35766 #35749 #35721 #35618 #35204 #40804
Thanks!

[tueboesen]

@tueboesen
Can you please refer to this comment for the cuda version and compatibility.

Please, check Your CPU/Python is on 32 bits?Please, refer #36167 and see if it helps you.
similar issues #21719 #36167 #36151 #36138 #36054 #36045 #36020 #36003 #35988 #35903 #35880 #35865 #35805 #35789 #35773 #35772 #35767 #35766 #35749 #35721 #35618 #35204 #40804
Thanks!

I have the right cuda version and compatibility, and my system is definitely 64 bit, including python. (I have managed to run tensorflow==1.4 just fine, so the problem seems to be with the gpu/cuda part of the setup.
I have an intel core i7 cpu, which can run AVX instructions (see CPU info below).
I haven't found anything else in there that suggests what my problem might be.

Coreinfo v3.5 - Dump information on system CPU and memory topology
Copyright (C) 2008-2020 Mark Russinovich
Sysinternals - www.sysinternals.com


Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
Intel64 Family 6 Model 142 Stepping 10, GenuineIntel
Microcode signature: 00000096
HTT             *       Hyperthreading enabled
HYPERVISOR      -       Hypervisor is present
VMX             *       Supports Intel hardware-assisted virtualization
SVM             -       Supports AMD hardware-assisted virtualization
X64             *       Supports 64-bit mode

SMX             -       Supports Intel trusted execution
SKINIT          -       Supports AMD SKINIT

NX              *       Supports no-execute page protection
SMEP            *       Supports Supervisor Mode Execution Prevention
SMAP            *       Supports Supervisor Mode Access Prevention
PAGE1GB         *       Supports 1 GB large pages
PAE             *       Supports > 32-bit physical addresses
PAT             *       Supports Page Attribute Table
PSE             *       Supports 4 MB pages
PSE36           *       Supports > 32-bit address 4 MB pages
PGE             *       Supports global bit in page tables
SS              *       Supports bus snooping for cache operations
VME             *       Supports Virtual-8086 mode
RDWRFSGSBASE    *       Supports direct GS/FS base access

FPU             *       Implements i387 floating point instructions
MMX             *       Supports MMX instruction set
MMXEXT          -       Implements AMD MMX extensions
3DNOW           -       Supports 3DNow! instructions
3DNOWEXT        -       Supports 3DNow! extension instructions
SSE             *       Supports Streaming SIMD Extensions
SSE2            *       Supports Streaming SIMD Extensions 2
SSE3            *       Supports Streaming SIMD Extensions 3
SSSE3           *       Supports Supplemental SIMD Extensions 3
SSE4a           -       Supports Streaming SIMDR Extensions 4a
SSE4.1          *       Supports Streaming SIMD Extensions 4.1
SSE4.2          *       Supports Streaming SIMD Extensions 4.2

AES             *       Supports AES extensions
AVX             *       Supports AVX instruction extensions
FMA             *       Supports FMA extensions using YMM state
MSR             *       Implements RDMSR/WRMSR instructions
MTRR            *       Supports Memory Type Range Registers
XSAVE           *       Supports XSAVE/XRSTOR instructions
OSXSAVE         *       Supports XSETBV/XGETBV instructions
RDRAND          *       Supports RDRAND instruction
RDSEED          *       Supports RDSEED instruction

CMOV            *       Supports CMOVcc instruction
CLFSH           *       Supports CLFLUSH instruction
CX8             *       Supports compare and exchange 8-byte instructions
CX16            *       Supports CMPXCHG16B instruction
BMI1            *       Supports bit manipulation extensions 1
BMI2            *       Supports bit manipulation extensions 2
ADX             *       Supports ADCX/ADOX instructions
DCA             -       Supports prefetch from memory-mapped device
F16C            *       Supports half-precision instruction
FXSR            *       Supports FXSAVE/FXSTOR instructions
FFXSR           -       Supports optimized FXSAVE/FSRSTOR instruction
MONITOR         *       Supports MONITOR and MWAIT instructions
MOVBE           *       Supports MOVBE instruction
ERMSB           *       Supports Enhanced REP MOVSB/STOSB
PCLMULDQ        *       Supports PCLMULDQ instruction
POPCNT          *       Supports POPCNT instruction
LZCNT           *       Supports LZCNT instruction
SEP             *       Supports fast system call instructions
LAHF-SAHF       *       Supports LAHF/SAHF instructions in 64-bit mode
HLE             -       Supports Hardware Lock Elision instructions
RTM             -       Supports Restricted Transactional Memory instructions

DE              *       Supports I/O breakpoints including CR4.DE
DTES64          *       Can write history of 64-bit branch addresses
DS              *       Implements memory-resident debug buffer
DS-CPL          *       Supports Debug Store feature with CPL
PCID            *       Supports PCIDs and settable CR4.PCIDE
INVPCID         *       Supports INVPCID instruction
PDCM            *       Supports Performance Capabilities MSR
RDTSCP          *       Supports RDTSCP instruction
TSC             *       Supports RDTSC instruction
TSC-DEADLINE    *       Local APIC supports one-shot deadline timer
TSC-INVARIANT   *       TSC runs at constant rate
xTPR            *       Supports disabling task priority messages

EIST            *       Supports Enhanced Intel Speedstep
ACPI            *       Implements MSR for power management
TM              *       Implements thermal monitor circuitry
TM2             *       Implements Thermal Monitor 2 control
APIC            *       Implements software-accessible local APIC
x2APIC          *       Supports x2APIC

CNXT-ID         -       L1 data cache mode adaptive or BIOS

MCE             *       Supports Machine Check, INT18 and CR4.MCE
MCA             *       Implements Machine Check Architecture
PBE             *       Supports use of FERR#/PBE# pin

PSN             -       Implements 96-bit processor serial number

PREFETCHW       *       Supports PREFETCHW instruction

Maximum implemented CPUID leaves: 00000016 (Basic), 80000008 (Extended).
Maximum implemented address width: 48 bits (virtual), 39 bits (physical).

Processor signature: 000806EA

Logical to Physical Processor Map:
**------  Physical Processor 0 (Hyperthreaded)
--**----  Physical Processor 1 (Hyperthreaded)
----**--  Physical Processor 2 (Hyperthreaded)
------**  Physical Processor 3 (Hyperthreaded)

Logical Processor to Socket Map:
********  Socket 0

Logical Processor to NUMA Node Map:
********  NUMA Node 0

No NUMA nodes.

Logical Processor to Cache Map:
**------  Data Cache          0, Level 1,   32 KB, Assoc   8, LineSize  64
**------  Instruction Cache   0, Level 1,   32 KB, Assoc   8, LineSize  64
**------  Unified Cache       0, Level 2,  256 KB, Assoc   4, LineSize  64
********  Unified Cache       1, Level 3,    8 MB, Assoc  16, LineSize  64
--**----  Data Cache          1, Level 1,   32 KB, Assoc   8, LineSize  64
--**----  Instruction Cache   1, Level 1,   32 KB, Assoc   8, LineSize  64
--**----  Unified Cache       2, Level 2,  256 KB, Assoc   4, LineSize  64
----**--  Data Cache          2, Level 1,   32 KB, Assoc   8, LineSize  64
----**--  Instruction Cache   2, Level 1,   32 KB, Assoc   8, LineSize  64
----**--  Unified Cache       3, Level 2,  256 KB, Assoc   4, LineSize  64
------**  Data Cache          3, Level 1,   32 KB, Assoc   8, LineSize  64
------**  Instruction Cache   3, Level 1,   32 KB, Assoc   8, LineSize  64
------**  Unified Cache       4, Level 2,  256 KB, Assoc   4, LineSize  64

[jvishnuvardhan]

@tueboesen Everything seems rights. Can you please share the commands you have used to install TF and GPU inside an environment. Thanks!

[tueboesen]

@tueboesen Everything seems rights. Can you please share the commands you have used to install TF and GPU inside an environment. Thanks!

I just went through the process once again, on my desktop computer dedicated for deep learning, just to verify the problem and I ended up with the same issue there.

I know this is a rather old version of tensorflow, and I wish I could just use a newer version but unfortunately, I need either version 1.13 or 1.14 in order to run the following code from a research article:
https://github.com/gjoni/trRosetta

My process was the following:

1. Install python 3.6.0 (python-3.6.0-amd64.exe)
2. Spin up a new virtual environment with python 3.6.0
3. pip install tensorflow-gpu==1.14
4. try to run: sess = tf.Session(), and get told that I'm missing a DLL, and to install CUDA 10.0
5. Install CUDA 10.0 (cuda_10.0.130_411.31_win10.exe)
6. try to run: sess = tf.Session(), and get told that I'm missing another DLL, related to cuDNN 7.
7. Install cudnn 7 (download cudnn-10.0-windows10-x64-v7.6.5.32.zip, extract it, and move it to the CUDA 10.0 folder)
8. try to run: sess = tf.Session(), and get the following error:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\Tue\PycharmProjects\trRosetta\venv_36\lib\site-packages\tensorflow\__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\Tue\PycharmProjects\trRosetta\venv_36\lib\site-packages\tensorflow\python\__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\Tue\PycharmProjects\trRosetta\venv_36\lib\site-packages\tensorflow\core\framework\graph_pb2.py", line 7, in <module>
    from google.protobuf import descriptor as _descriptor
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "C:\Users\Tue\PycharmProjects\trRosetta\venv_36\lib\site-packages\google\protobuf\descriptor.py", line 48, in <module>
    from google.protobuf.pyext import _message
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified procedure could not be found.

Additional info on this desktop system:

Coreinfo v3.5 - Dump information on system CPU and memory topology
Copyright (C) 2008-2020 Mark Russinovich
Sysinternals - www.sysinternals.com


AMD Ryzen 7 2700X Eight-Core Processor
AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD
Microcode signature: 00000000
HTT             *       Multicore
HYPERVISOR      *       Hypervisor is present
VMX             -       Supports Intel hardware-assisted virtualization
SVM             -       Supports AMD hardware-assisted virtualization
X64             *       Supports 64-bit mode

SMX             -       Supports Intel trusted execution
SKINIT          -       Supports AMD SKINIT

NX              *       Supports no-execute page protection
SMEP            *       Supports Supervisor Mode Execution Prevention
SMAP            *       Supports Supervisor Mode Access Prevention
PAGE1GB         *       Supports 1 GB large pages
PAE             *       Supports > 32-bit physical addresses
PAT             *       Supports Page Attribute Table
PSE             *       Supports 4 MB pages
PSE36           *       Supports > 32-bit address 4 MB pages
PGE             *       Supports global bit in page tables
SS              -       Supports bus snooping for cache operations
VME             *       Supports Virtual-8086 mode
RDWRFSGSBASE    *       Supports direct GS/FS base access

FPU             *       Implements i387 floating point instructions
MMX             *       Supports MMX instruction set
MMXEXT          *       Implements AMD MMX extensions
3DNOW           -       Supports 3DNow! instructions
3DNOWEXT        -       Supports 3DNow! extension instructions
SSE             *       Supports Streaming SIMD Extensions
SSE2            *       Supports Streaming SIMD Extensions 2
SSE3            *       Supports Streaming SIMD Extensions 3
SSSE3           *       Supports Supplemental SIMD Extensions 3
SSE4a           *       Supports Streaming SIMDR Extensions 4a
SSE4.1          *       Supports Streaming SIMD Extensions 4.1
SSE4.2          *       Supports Streaming SIMD Extensions 4.2

AES             *       Supports AES extensions
AVX             *       Supports AVX instruction extensions
FMA             *       Supports FMA extensions using YMM state
MSR             *       Implements RDMSR/WRMSR instructions
MTRR            *       Supports Memory Type Range Registers
XSAVE           *       Supports XSAVE/XRSTOR instructions
OSXSAVE         *       Supports XSETBV/XGETBV instructions
RDRAND          *       Supports RDRAND instruction
RDSEED          *       Supports RDSEED instruction

CMOV            *       Supports CMOVcc instruction
CLFSH           *       Supports CLFLUSH instruction
CX8             *       Supports compare and exchange 8-byte instructions
CX16            *       Supports CMPXCHG16B instruction
BMI1            *       Supports bit manipulation extensions 1
BMI2            *       Supports bit manipulation extensions 2
ADX             *       Supports ADCX/ADOX instructions
DCA             -       Supports prefetch from memory-mapped device
F16C            *       Supports half-precision instruction
FXSR            *       Supports FXSAVE/FXSTOR instructions
FFXSR           *       Supports optimized FXSAVE/FSRSTOR instruction
MONITOR         -       Supports MONITOR and MWAIT instructions
MOVBE           *       Supports MOVBE instruction
ERMSB           -       Supports Enhanced REP MOVSB/STOSB
PCLMULDQ        *       Supports PCLMULDQ instruction
POPCNT          *       Supports POPCNT instruction
LZCNT           *       Supports LZCNT instruction
SEP             *       Supports fast system call instructions
LAHF-SAHF       *       Supports LAHF/SAHF instructions in 64-bit mode
HLE             -       Supports Hardware Lock Elision instructions
RTM             -       Supports Restricted Transactional Memory instructions

DE              *       Supports I/O breakpoints including CR4.DE
DTES64          -       Can write history of 64-bit branch addresses
DS              -       Implements memory-resident debug buffer
DS-CPL          -       Supports Debug Store feature with CPL
PCID            -       Supports PCIDs and settable CR4.PCIDE
INVPCID         -       Supports INVPCID instruction
PDCM            -       Supports Performance Capabilities MSR
RDTSCP          *       Supports RDTSCP instruction
TSC             *       Supports RDTSC instruction
TSC-DEADLINE    -       Local APIC supports one-shot deadline timer
TSC-INVARIANT   *       TSC runs at constant rate
xTPR            -       Supports disabling task priority messages

EIST            -       Supports Enhanced Intel Speedstep
ACPI            -       Implements MSR for power management
TM              -       Implements thermal monitor circuitry
TM2             -       Implements Thermal Monitor 2 control
APIC            *       Implements software-accessible local APIC
x2APIC          -       Supports x2APIC

CNXT-ID         -       L1 data cache mode adaptive or BIOS

MCE             *       Supports Machine Check, INT18 and CR4.MCE
MCA             *       Implements Machine Check Architecture
PBE             -       Supports use of FERR#/PBE# pin

PSN             -       Implements 96-bit processor serial number

PREFETCHW       *       Supports PREFETCHW instruction

Maximum implemented CPUID leaves: 0000000D (Basic), 8000001E (Extended).
Maximum implemented address width: 48 bits (virtual), 48 bits (physical).

Processor signature: 00800F82

GPU: Nvidia Geforce RTX 2080 Ti

A quick pip list on the system gives the following:

Microsoft Windows [Version 10.0.18362.900]
(c) 2019 Microsoft Corporation. All rights reserved.

(venv_36) C:\Users\Tue\PycharmProjects\trRosetta>pip list
Package              Version
-------------------- -------
absl-py              0.9.0
astor                0.8.1
gast                 0.4.0
google-pasta         0.2.0
grpcio               1.31.0
h5py                 2.10.0
importlib-metadata   1.7.0
Keras-Applications   1.0.8
Keras-Preprocessing  1.1.2
Markdown             3.2.2
numpy                1.19.1
pip                  20.2.2
protobuf             3.12.4
setuptools           49.3.1
six                  1.15.0
tensorboard          1.14.0
tensorflow-estimator 1.14.0
tensorflow-gpu       1.14.0
termcolor            1.1.0
Werkzeug             1.0.1
wheel                0.34.2
wrapt                1.12.1
zipp                 3.1.0

(venv_36) C:\Users\Tue\PycharmProjects\trRosetta>

Finally it should be noted that I am able to run deep learning codes on both these computers, although they use newer versions of these libraries. I have both cuda 10.1 and 10.0 install right now, but 10.0 is the default in the $PATH environment at the moment due to this.

[gunan]

There may be various reasons, like your python may be 32 bit, or on windows, we saw that anything downloaded from windows store has issues due to sandboxing, or missing visual studio redistributable.
However, 1.14 is well outside our support windows, so I will mark this as community support.

[Saduf2019]

@tueboesen
We see that this issue is for 1.x which is not currently supported, could you please upgrade to 2.x and let us know.
Please refer to this guide for the compatibility.

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No