[A64FX][POCL][SYCL] I am getting an error compiling SYCL programs with POCL backend.

[HariAbram]

Hi, I am linking SYCL with POCL on an A64FX machine. I managed to get it to work, and the SYCL program compile without an issue but I am encountering runtime issues.

** Final POCL_DEBUG flags: FFFFFFFFFFFFFFFF 
[2024-05-09 08:20:21.850763600]POCL: in fn pocl_read_file at line 173:
  |     ERROR |  fopen( /sys/devices/system/cpu/cpu0/cache/index3/size ) failed
[2024-05-09 08:20:21.850921991]POCL: in fn pocl_read_file at line 173:
  |     ERROR |  fopen( /sys/devices/system/cpu/cpu0/cache/index2/size ) failed
[2024-05-09 08:20:21.281471532696932]POCL: in fn pocl_topology_detect_device_info at line 244:
  |   WARNING |  Could not figure out CPU cache size, using bogus value
[2024-05-09 08:20:21.281471540269137]POCL: in fn POclCreateCommandQueue at line 104:
  |   GENERAL |  Created Command Queue 3 (0xf1b1470) on device 0
[2024-05-09 08:20:21.473305019567]POCL: in fn void pocl_llvm_create_context(cl_context) at line 435:
  |      LLVM |  Created context 2 (0xf1b0820)
[2024-05-09 08:20:21.4616194021254885068]POCL: in fn POclCreateContext at line 233:
  |   GENERAL |  Created Context 2 (0xf1b0820)
[2024-05-09 08:20:21.281471540438158]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0xf1b0820), Refcount: 2
[2024-05-09 08:20:21.281471540465928]POCL: in fn POclCreateCommandQueue at line 104:
  |   GENERAL |  Created Command Queue 4 (0xf1b3890) on device 0
[2024-05-09 08:20:21.281471540679511]POCL: in fn POclGetExtensionFunctionAddressForPlatform at line 151:
  |     ERROR |  unknown platform extension requested: clCreateBufferWithPropertiesINTEL
[2024-05-09 08:20:21.281471540717181]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0xf1b0820), Refcount: 3
[2024-05-09 08:20:21.4503600486368247]POCL: in fn POclCreateBuffer at line 261:
  |    MEMORY |  Created Buffer 5 (0xf1b5b40), MEM_HOST_PTR: (nil), device_ptrs[0]: (nil), SIZE 32, FLAGS 1 
[2024-05-09 08:20:21.859123472]POCL: in fn POclRetainContext at line 33:
  | REFCOUNTS |  Retain Context 2 (0xf1b0820), Refcount: 4
[2024-05-09 08:20:21.281471541988573]POCL: in fn pocl_run_command_capture_output at line 2007:
  |   GENERAL |  Launching: /home/hari.abram/sycl_workspace/llvm/build/install/bin/llvm-spirv
[2024-05-09 08:20:21.912044685]POCL: in fn compile_and_link_program at line 708:
  |      LLVM |  building program with options 
[2024-05-09 08:20:21.912093215]POCL: in fn compile_and_link_program at line 745:
  |      LLVM |  building program for 1 devs with options 
[2024-05-09 08:20:21.912120256]POCL: in fn compile_and_link_program at line 749:
  |      LLVM |     BUILDING for device: cpu
[2024-05-09 08:20:21.913188866]POCL: in fn pocl_llvm_convert_and_link_ir at line 635:
  |      LLVM |  Cached compiled SPIRV binary not found, generating SPIR IR to /home/hari.abram/.cache/pocl/kcache/HF/FGIMNMLJICNOFBJAJJCMBJNBGIKFEBBAJEACJ/program.bc
[2024-05-09 08:20:21.281471596272479]POCL: in fn pocl_run_command at line 1956:
  |   GENERAL |  Launching: /home/hari.abram/sycl_workspace/llvm/build/install/bin/llvm-spirv
[2024-05-09 08:20:21.281471657470444]POCL: in fn llvm::Module *getKernelLibrary(cl_device_id, PoclLLVMContextData *) at line 1067:
  |      LLVM |  Using /home/hari.abram/POCL/install/pocl/lib64/../share/pocl/kernel-aarch64-linux-gnu-a64fx.bc as the built-in lib.
[2024-05-09 08:20:23.707081036]POCL: in fn int pocl_llvm_link_program(cl_program, unsigned int, cl_uint, unsigned char **, size_t *, void **, int, int) at line 858:
  |     ERROR | CL_LINK_PROGRAM_FAILURE could not parse module
[2024-05-09 08:20:23.281471388884707]POCL: in fn pocl_llvm_convert_and_link_ir at line 693:
  |     ERROR | CL_LINK_PROGRAM_FAILURE Failed to link program.bc
[2024-05-09 08:20:23.707175887]POCL: in fn compile_and_link_program at line 874:
  |     ERROR | CL_BUILD_PROGRAM_FAILURE Device cpu-0x001 failed to build the program
[2024-05-09 08:20:23.281471389111569]POCL: in fn POclReleaseProgram at line 52:
  | REFCOUNTS |  Release Program 6 (0xf1b6770), Refcount: 0, Kernel #: 0 
[2024-05-09 08:20:23.707403110]POCL: in fn POclReleaseProgram at line 62:
  | REFCOUNTS |  Free Program 6 (0xf1b6770)
[2024-05-09 08:20:23.707479480]POCL: in fn POclReleaseContext at line 49:
  | REFCOUNTS |  Release Context 2 (0xf1b0820), Refcount: 3
terminate called after throwing an instance of 'sycl::_V1::compile_program_error'
  what():  The program was built for 1 devices
Build program log for 'cpu-0x001':
Device cpu-0x001 failed to build the program
 -11 (PI_ERROR_BUILD_PROGRAM_FAILURE)
Aborted (core dumped)

Does anyone have any idea what could be causing this issue? this issue seems to be from the POCL end.

[franz]

Hello, could you try if this patch fixes it ?

[HariAbram]

I tried it, and I am still getting the same errors (Could not parse module).

[pjaaskel]

My quick guess is that there's a mixup of LLVM/SPIR-V toolchains somehow: You utilize LLVM-SPIRV from the Intel LLVM and perhaps an upstream LLVM and then the Intel LLVM-SPIRV produces unknown (debug) data or similar. Or the other way around: PoCL's bitcode lib has been built with upstream (system) LLVM and it's not compatible (perhaps based on a newer LLVM version) with the Intel LLVM installed with your SYCL frontend.

[franz]

Did you try cleaning the pocl kernel cache ? Also what pjaaskel says about version mixup is quite possible. The parse error means that LLVM is unable to parse the in-memory LLVM IR bitstream into a valid llvm::Module object. So either the content is completely wrong and not LLVM IR at all, or it is LLVM IR but has attributes the parser doesn't recognize.

[HariAbram]

@pjaaskel is right. I got it to working when I used newer versions of all the tool chains. I manged to compile and run a simple SYCL program. I am using POCL v6.0-pre main-0-gf8711b0e compiled with llvm v17.0.6 and SYCL with Intel llvm v19.0.0. Thank you for your help.