Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error run train in gpu #106

Open
biexiangduo opened this issue Jun 30, 2019 · 1 comment
Open

error run train in gpu #106

biexiangduo opened this issue Jun 30, 2019 · 1 comment

Comments

@biexiangduo
Copy link

cuda:8.0
cudnn:6.0
python:3.5
tensorflow:1.14.0
~/CycleGAN-TensorFlow$ python3 train.py
WARNING: Logging before flag parsing goes to stderr.
W0630 08:35:53.787037 140031590201088 deprecation_wrapper.py:119] From train.py:135: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

W0630 08:35:53.796201 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:49: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

W0630 08:35:53.797244 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:58: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0630 08:35:53.798269 140031590201088 deprecation.py:323] From /home/pf/CycleGAN-TensorFlow/reader.py:19: TFRecordReader.init (from tensorflow.python.ops.io_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by tf.data. Use tf.data.TFRecordDataset.
W0630 08:35:53.799142 140031590201088 deprecation.py:323] From /home/pf/CycleGAN-TensorFlow/reader.py:28: string_input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensor_slices(string_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). If shuffle=False, omit the .shuffle(...).
W0630 08:35:53.803668 140031590201088 deprecation.py:323] From /home/pf/.local/lib/python3.5/site-packages/tensorflow/python/training/input.py:278: input_producer (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensor_slices(input_tensor).shuffle(tf.shape(input_tensor, out_type=tf.int64)[0]).repeat(num_epochs). If shuffle=False, omit the .shuffle(...).
W0630 08:35:53.804200 140031590201088 deprecation.py:323] From /home/pf/.local/lib/python3.5/site-packages/tensorflow/python/training/input.py:190: limit_epochs (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.from_tensors(tensor).repeat(num_epochs).
W0630 08:35:53.805154 140031590201088 deprecation.py:323] From /home/pf/.local/lib/python3.5/site-packages/tensorflow/python/training/input.py:199: QueueRunner.init (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
W0630 08:35:53.805793 140031590201088 deprecation.py:323] From /home/pf/.local/lib/python3.5/site-packages/tensorflow/python/training/input.py:199: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
W0630 08:35:53.808638 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/reader.py:32: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

W0630 08:35:53.808758 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/reader.py:35: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

W0630 08:35:53.810475 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/reader.py:52: The name tf.image.resize_images is deprecated. Please use tf.image.resize instead.

W0630 08:35:53.814553 140031590201088 deprecation.py:323] From /home/pf/CycleGAN-TensorFlow/reader.py:45: shuffle_batch (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.shuffle(min_after_dequeue).batch(batch_size).
W0630 08:35:53.820895 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/reader.py:48: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0630 08:35:53.839401 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/generator.py:21: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0630 08:35:53.839734 140031590201088 deprecation.py:506] From /home/pf/CycleGAN-TensorFlow/ops.py:188: calling RandomNormal.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0630 08:35:55.420766 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:87: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead.

W0630 08:35:55.840024 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:92: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0630 08:35:56.864042 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:119: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0630 08:35:56.867225 140031590201088 deprecation.py:323] From /home/pf/.local/lib/python3.5/site-packages/tensorflow/python/keras/optimizer_v2/learning_rate_schedule.py:409: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
W0630 08:35:56.869565 140031590201088 deprecation.py:323] From /home/pf/CycleGAN-TensorFlow/model.py:122: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0630 08:35:56.870930 140031590201088 deprecation_wrapper.py:119] From /home/pf/CycleGAN-TensorFlow/model.py:129: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0630 08:36:08.143354 140031590201088 deprecation_wrapper.py:119] From train.py:67: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

W0630 08:36:08.144193 140031590201088 deprecation_wrapper.py:119] From train.py:68: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

W0630 08:36:10.731404 140031590201088 deprecation_wrapper.py:119] From train.py:69: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

2019-06-30 08:36:10.979949: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-06-30 08:36:11.179523: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-30 08:36:11.179791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.733
pciBusID: 0000:01:00.0
2019-06-30 08:36:11.179861: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.179906: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.179954: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.179997: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.180039: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.180081: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.180122: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda-8.0/lib64:
2019-06-30 08:36:11.180131: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-06-30 08:36:11.180360: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-06-30 08:36:11.221799: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-06-30 08:36:11.222076: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x12626b60 executing computations on platform CUDA. Devices:
2019-06-30 08:36:11.222090: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce GTX 1060, Compute Capability 6.1
2019-06-30 08:36:11.243575: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3192000000 Hz
2019-06-30 08:36:11.244643: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1239d550 executing computations on platform Host. Devices:
2019-06-30 08:36:11.244657: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
2019-06-30 08:36:11.244700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-06-30 08:36:11.244709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]
2019-06-30 08:36:12.565100: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
W0630 08:36:12.775092 140031590201088 deprecation.py:323] From train.py:83: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.


could I change cudnn or tensorflow or python version to run in gpu?

@rkfg
Copy link
Contributor

rkfg commented Jul 11, 2019

You need to update CUDA to version 10. Don't forget to change $LD_LIBRARY_PATH, I suggest using a symlink /usr/local/cuda pointing to the latest version so it's easier to update (the CUDA installer does this for you usually), my $LD_LIBRARY_PATH is /usr/local/cuda/lib64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants