Why is faceswap is not using GPU?

[shadowzoom]

As you can see by GPU's graph on the right side of screen, GPU is even not reacting normally... While all cpu cores working at 100%
Video proof: https://streamable.com/jj75w

[Clorr]

Have you installed tensorflow-gpu? Your log does not show the gpu initialization

[Clorr]

What is your monitoring app? It seems quite pretty ;-)

[shadowzoom]

Installed tensorflow-gpu, but i'm getting:
python3 faceswap.py train -A '/home/tuw/Desktop/faceswap/data/trump1' -B '/home/tuw/Desktop/faceswap/data/cage' -m '/home/tuw/Desktop/faceswap/data/model' -p
Model A Directory: /home/tuw/Desktop/faceswap/data/trump1
Model B Directory: /home/tuw/Desktop/faceswap/data/cage
Training data directory: /home/tuw/Desktop/faceswap/data/model
Loading data, this may take a while...
Using live preview
Loading Model from Model_Original plugin...
/home/tuw/.local/lib/python3.5/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.5/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/home/tuw/Desktop/faceswap/scripts/train.py", line 122, in processThread
model = PluginLoader.get_model(trainer)(self.arguments.model_dir)
File "/home/tuw/Desktop/faceswap/plugins/PluginLoader.py", line 13, in get_model
return PluginLoader.import("Model", "Model{0}".format(name))
File "/home/tuw/Desktop/faceswap/plugins/PluginLoader.py", line 22, in _import
module = import(name, globals(), locals(), [], 1)
File "/home/tuw/Desktop/faceswap/plugins/Model_Original.py", line 3, in
from keras.models import Model as KerasModel
File "/home/tuw/.local/lib/python3.5/site-packages/keras/init.py", line 3, in
from . import utils
File "/home/tuw/.local/lib/python3.5/site-packages/keras/utils/init.py", line 6, in
from . import conv_utils
File "/home/tuw/.local/lib/python3.5/site-packages/keras/utils/conv_utils.py", line 9, in
from .. import backend as K
File "/home/tuw/.local/lib/python3.5/site-packages/keras/backend/init.py", line 83, in
from .tensorflow_backend import *
File "/home/tuw/.local/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 5, in
import tensorflow as tf
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/init.py", line 24, in
from tensorflow.python import *
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/init.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/tuw/.local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.5/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.5/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.

Why is that happening? I already have nvidia driver and cuda8 installed
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

And in ~/.bashrc i have path's:
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

P.S. Monitoring app is conky, here is my ~/.conyrc file https://pastebin.com/5fnPAx6m

[fat-tire]

do you have cudnn installed?

Also your export looks strange in in ~/.bashrc. Here's what I added:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64

Finally, see here.

[shadowzoom]

@fat-tire yes, i have cudnn for cuda8.
I thought it was because tensorflow 1.5, i've reinstalled tensorflow 1.4 (pip3 install --upgrade tensorflow-gpu==1.4)

Added export path like you
#cuda 8
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64

But... Now i have ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory
Hm... Why is showing libcudnn.so.6?

[fat-tire]

I'm not sure-- could it be baked into the tensorflow you installed with pip? You really might consider building tensorflow yourself. I'm running CUDA 9.1 and CudNN 7.... and it works with a self-built tensorflow. that way duringthe installation I could tell it exactly what the version of everything is. I'm writing up some tips right now on self-building for Linu FWIW so I'll post those in acouple hours in a PR.

[shadowzoom]

Thanks mate! I fix it by installed and add libcudnn6-dev version:

https://yangcha.github.io/Install-CUDA8/
p.s. Are you now running tensorflow 1.5 with CUDA 9.1? Would be great if you paste here your pip packages (pip3 list)

[fat-tire]

Glad it's working... yes I am running 1.5 (master branch) w/CUDA 9.1. I posted the pip packages here and I'm going to post an more extensive LINUX-README.md that I'll write later today.