OpenCL GPU support issue

[Kenop]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04 LTS (Bionic Beaver)
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): v1.8.0-2217-g7ad3964dcb 1.8.0
• Python version: Python 3.6.5
• Bazel version (if compiling from source): Build label: 0.13.1
• GCC/Compiler version (if compiling from source): g++ (Ubuntu 7.3.0-16ubuntu3) 7.3.0
• CUDA/cuDNN version: NA
• GPU model and memory: AMD Radeon R9290
• Exact command to reproduce: bazel test -c opt --config=sycl --test_output=all //tensorflow/python/kernel_tests:basic_gpu_test

Describe the problem

I built TF 1.8 from source with 'y' for OpenCL SYCL and ComputeCPP support in build configuration ('n' for other options). Build was complete successfully, but GPU device seems unavailable in TF, only CPU:0 is listed by
tensorflow.python.client.device_lib.device_lib.list_local_devices()
and
tensorflow.contrib.eager.list_devices()
Also, tensorflow.matmul(a, b) is placed on CPU:0, log_device_placement says: "Device mapping: no known devices."

bazel test -c opt --config=sycl --test_output=all //tensorflow/python/kernel_tests:basic_gpu_test
fails with errors from SYCL.

Bazel test, clinfo and computecpp_info logs are listed below.

Requirements for OpenCL GPU support seem to be met, but it doesn't work. Am I missing something?

Source code / logs

clinfo

Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
Platform Version OpenCL 2.1 AMD-APP (2633.3)
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
Platform Host timer resolution 1ns
Platform Extensions function suffix AMD
Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name Hawaii
Device Vendor Advanced Micro Devices, Inc.
Device Vendor ID 0x1002
Device Version OpenCL 1.2 AMD-APP (2633.3)
Driver Version 2633.3
Device OpenCL C Version OpenCL C 1.2
Device Type GPU
Device Board Name (AMD) AMD Radeon R9 200 Series
Device Topology (AMD) PCI-E, 01:00.0
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 40
SIMD per compute unit (AMD) 4
SIMD width (AMD) 16
SIMD instruction width (AMD) 1
Max clock frequency 947MHz
Graphics IP (AMD) 7.2
Device Partition (core)
Max number of sub-devices 40
Supported partition types none specified
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 256
Preferred work group size (AMD) 256
Max work group size (AMD) 1024
Preferred work group size multiple 64
Wavefront width (AMD) 64
Preferred / native vector sizes
char 4 / 4
short 2 / 2
int 1 / 1
long 1 / 1
half 1 / 1 (n/a)
float 1 / 1
double 1 / 1 (cl_khr_fp64)
Half-precision Floating-point support (n/a)
Single-precision Floating-point support (core)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 2471116800 (2.301GiB)
Global free memory (AMD) 2398160 (2.287GiB)
Global memory channels (AMD) 16
Global memory banks per channel (AMD) 16
Global memory bank width (AMD) 256 bytes
Error Correction support No
Max memory allocation 2082456371 (1.939GiB)
Unified memory for Host and Device No
Minimum alignment for any data type 128 bytes
Alignment of base address 2048 bits (256 bytes)
Global Memory cache type Read/Write
Global Memory cache size 16384 (16KiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 16
Max size for 1D images from buffer 134217728 pixels
Max 1D or 2D image array size 2048 images
Base address alignment for 2D image buffers 256 bytes
Pitch alignment for 2D image buffers 256 pixels
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 8
Local memory type Local
Local memory size 32768 (32KiB)
Local memory syze per CU (AMD) 65536 (64KiB)
Local memory banks (AMD) 32
Max number of constant args 8
Max constant buffer size 2082456371 (1.939GiB)
Preferred constant buffer size (AMD) 16384 (16KiB)
Max size of kernel argument 1024
Queue properties
Out-of-order execution No
Profiling Yes
Prefer user sync for interop Yes
Profiling timer resolution 1ns
Profiling timer offset since Epoch (AMD) 1527256350447623220ns (Fri May 25 10:52:30 2018)
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
Thread trace supported (AMD) Yes
Number of async queues (AMD) 2
Max real-time compute queues (AMD) 0
Max real-time compute units (AMD) 0
SPIR versions 1.2
printf() buffer size 4194304 (4MiB)
Built-in kernels
Device Extensions cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_khr_spir cl_khr_gl_event
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) No platform
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) No platform
clCreateContext(NULL, ...) [default] No platform
clCreateContext(NULL, ...) [other] Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name Hawaii
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name Hawaii
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name Hawaii

/usr/local/computecpp/bin/computecpp_info

ComputeCpp Info (CE 0.8.0)
Toolchain information:
GLIBC version: 2.27
GLIBCXX: 20160609
This version of libstdc++ is supported.
Device Info:
Discovered 1 devices matching:
platform :
device type :
Device 0:
Device is supported : UNTESTED - Untested OS
CL_DEVICE_NAME : Hawaii
CL_DEVICE_VENDOR : Advanced Micro Devices, Inc.
CL_DRIVER_VERSION : 2633.3
CL_DEVICE_TYPE : CL_DEVICE_TYPE_GPU

bazel test -c opt --config=sycl --test_output=all //tensorflow/python/kernel_tests:basic_gpu_test

INFO: Analysed target //tensorflow/python/kernel_tests:basic_gpu_test (0 packages loaded).
INFO: Found 1 test target...
ERROR: /home/dmitry/Soft/dev/source/TensorFlow/tensorflow/tensorflow/core/BUILD:715:1: C++ compilation of rule '//tensorflow/core:ctc_ops_op_lib' failed (Exit 1)
In file included from tensorflow/core/ops/ctc_ops.cc:17:
In file included from ./tensorflow/core/framework/shape_inference.h:20:
In file included from ./tensorflow/core/framework/node_def_util.h:23:
In file included from ./tensorflow/core/framework/attr_value_util.h:23:
In file included from ./tensorflow/core/framework/partial_tensor_shape.h:20:
In file included from ./tensorflow/core/framework/tensor_shape.h:21:
In file included from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:
In file included from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:99:
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:163:19: error: no matching member function for call to 'get_access'
auto ptr =buf.get_access<cl::sycl::access::mode::discard_write, cl::sycl::access::target::host_buffer>().get_pointer();
~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:593:3: note: candidate template ignored: invalid explicitly-specified argument for template parameter 'accessMode'
get_access() {
^
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:611:53: note: candidate function template not viable: requires single argument 'cgh', but no arguments were provided
accessor<T, dimensions, accessMode, accessTarget> get_access(
^
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:633:53: note: candidate function template not viable: requires 3 arguments, but 0 were provided
accessor<T, dimensions, accessMode, accessTarget> get_access(
^
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:653:53: note: candidate function template not viable: requires at least 2 arguments, but 0 were provided
accessor<T, dimensions, accessMode, accessTarget> get_access(
^
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:670:68: note: candidate function template not viable: requires 2 arguments, but 0 were provided
accessor<T, dimensions, accessMode, access::target::host_buffer> get_access(
^
external/local_config_sycl/crosstool/../sycl/include/SYCL/buffer.h:684:68: note: candidate function template not viable: requires at least argument 'range', but no arguments were provided
accessor<T, dimensions, accessMode, access::target::host_buffer> get_access(
^
In file included from tensorflow/core/ops/ctc_ops.cc:17:
In file included from ./tensorflow/core/framework/shape_inference.h:20:
In file included from ./tensorflow/core/framework/node_def_util.h:23:
In file included from ./tensorflow/core/framework/attr_value_util.h:23:
In file included from ./tensorflow/core/framework/partial_tensor_shape.h:20:
In file included from ./tensorflow/core/framework/tensor_shape.h:21:
In file included from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:
In file included from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:99:
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:195:61: error: no member named 'map_allocator' in namespace 'cl::sycl'
auto src_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(static_cast<void*>(const_cast<Index*>(src))), cl::sycl::range<1>(n));
~~~~~~~~~~^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:195:75: error: unexpected type name 'uint8_t': expected expression
auto src_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(static_cast<void*>(const_cast<Index*>(src))), cl::sycl::range<1>(n));
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:195:86: warning: expression result unused [-Wunused-value]
auto src_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(static_cast<void*>(const_cast<Index*>(src))), cl::sycl::range<1>(n));
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:198:31: error: expected unqualified-id
auto src_acc =src_buf.template get_access<cl::sycl::access::mode::read, cl::sycl::access::target::global_buffer>(cgh);
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:198:107: error: nested name specifier 'cl::sycl::access::target::' for declaration does not refer into a class, class template or class template partial specialization
auto src_acc =src_buf.template get_access<cl::sycl::access::mode::read, cl::sycl::access::target::global_buffer>(cgh);
~~~~~~~~~~~~~~~~~~~~~~~~~~^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:198:120: error: expected ';' at end of declaration
auto src_acc =src_buf.template get_access<cl::sycl::access::mode::read, cl::sycl::access::target::global_buffer>(cgh);
^
;
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:217:62: error: no member named 'map_allocator' in namespace 'cl::sycl'
auto dest_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(dst), cl::sycl::range<1>(n));
~~~~~~~~~~^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:217:76: error: unexpected type name 'uint8_t': expected expression
auto dest_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(dst), cl::sycl::range<1>(n));
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:217:87: warning: expression result unused [-Wunused-value]
auto dest_buf = cl::sycl::buffer<uint8_t, 1, cl::sycl::map_allocator<uint8_t> >(static_cast<uint8_t*>(dst), cl::sycl::range<1>(n));
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:220:32: error: expected unqualified-id
auto dst_acc =dest_buf.template get_access<cl::sycl::access::mode::discard_write, cl::sycl::access::target::global_buffer>(cgh);
^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:220:117: error: nested name specifier 'cl::sycl::access::target::' for declaration does not refer into a class, class template or class template partial specialization
auto dst_acc =dest_buf.template get_access<cl::sycl::access::mode::discard_write, cl::sycl::access::target::global_buffer>(cgh);
~~~~~~~~~~~~~~~~~~~~~~~~~~^
external/eigen_archive/unsupported/Eigen/CXX11/src/Tensor/TensorDeviceSycl.h:220:130: error: expected ';' at end of declaration
auto dst_acc =dest_buf.template get_access<cl::sycl::access::mode::discard_write, cl::sycl::access::target::global_buffer>(cgh);
^
;
2 warnings and 11 errors generated.
Target //tensorflow/python/kernel_tests:basic_gpu_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 10.427s, Critical Path: 9.81s
INFO: 4 processes, local.
FAILED: Build did NOT complete successfully
Executed 0 out of 1 test: 1 fails to build

[gunan]

I would recommend reaching out to codeplay. OpenCL support for TF is still in early stages, and there may be some additional constraints to use it.

[enihcam]

OpenCL support is integrating into 1.8:
https://github.com/lukeiwanski/tensorflow/tree/integration/1.8

[Harshini-Gadige]

@Kenop - Is this still an issue ? Feel free to close if this doesn't exist with the latest tensorflow version.

[Rbiessy]

Just for reference, this branch with TensorFlow 1.8 was only experimental.
The official branch is now using TensorFlow 1.9 here: https://github.com/codeplaysoftware/tensorflow/tree/eigen_sycl
It requires bazel 0.16 and the latest ComputeCpp version.

[gunan]

Closing as duplicate of #22