Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clang omp-target cannot select error. #1274

Open
rchen20 opened this issue May 26, 2022 · 8 comments · May be fixed by #1400
Open

Clang omp-target cannot select error. #1274

rchen20 opened this issue May 26, 2022 · 8 comments · May be fixed by #1400
Assignees
@rchen20
Labels
reviewed Mark with this label when issue has been discussed by team

Comments

@rchen20
Copy link
Member

rchen20 commented May 26, 2022 •
edited

When compiling for OpenMP Target with clang/ibm-13.0.1, there is clang backend error found by @kab163 :

[ 37%] Building CXX object test/functional/kernel/single-loop-tile-icount-tcount/CMakeFiles/test-kernel-single-loop-ForICount-8-OpenMPTarget.exe.dir/test-kernel-single-loop-ForICount-8-OpenMPTarget.cpp.o
cd /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/test/functional/kernel/single-loop-tile-icount-tcount && /usr/tce/packages/clang/clang-ibm-13.0.1/bin/clang++  -DCAMP_ENABLE_TARGET_OPENMP -DGTEST_HAS_DEATH_TEST=1 -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/test/include -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/test/functional/kernel/single-loop-tile-icount-tcount/tests -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/include -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/tpl/camp/include -I/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/tpl/camp/include -isystem /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include  -Wall -Wextra      -O3 -fPIE   -fopenmp -fopenmp-targets=nvptx64-nvidia-cuda -std=c++14 -o CMakeFiles/test-kernel-single-loop-ForICount-8-OpenMPTarget.exe.dir/test-kernel-single-loop-ForICount-8-OpenMPTarget.cpp.o -c /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/test/functional/kernel/single-loop-tile-icount-tcount/test-kernel-single-loop-ForICount-8-OpenMPTarget.cpp
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/test/functional/kernel/single-loop-tile-icount-tcount/test-kernel-single-loop-ForICount-8-OpenMPTarget.cpp:11:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/test/include/RAJA_test-base.hpp:15:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/RAJA.hpp:31:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/util/Operators.hpp:34:
/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/util/macros.hpp:133:16: warning: variable 'errtemp' set but not used [-Wunused-but-set-variable]
  const char * errtemp = nullptr;
               ^
1 warning generated.
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/build_lc_blueos-clang-ibm-13.0.1_omptarget/test/functional/kernel/single-loop-tile-icount-tcount/test-kernel-single-loop-ForICount-8-OpenMPTarget.cpp:11:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/test/include/RAJA_test-base.hpp:15:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/RAJA.hpp:31:
In file included from /usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/util/Operators.hpp:34:
/usr/WS1/chen59/allraja/rajaomptarget/raja_git_omptarget/include/RAJA/util/macros.hpp:133:16: warning: variable 'errtemp' set but not used [-Wunused-but-set-variable]
  const char * errtemp = nullptr;
               ^
fatal error: error in backend: Cannot select: 0x1c2e0c60: i16,ch = AtomicLoad<(dereferenceable load acquire (s8) from `i8 addrspace(1)* bitcast (i64 addrspace(1)* @_ZGVZN4camp9resources2v14Host11get_defaultEvE1h to i8 addrspace(1)*)`, align 8, addrspace 1)> 0x1c2e0cc8, 0x1c24d3f8
  0x1c24d3f8: i64 = NVPTXISD::Wrapper TargetGlobalAddress:i64<i64 addrspace(1)* @_ZGVZN4camp9resources2v14Host11get_defaultEvE1h> 0
    0x1c24d188: i64 = TargetGlobalAddress<i64 addrspace(1)* @_ZGVZN4camp9resources2v14Host11get_defaultEvE1h> 0
In function: __omp_outlined__1
clang-13: error: clang frontend command failed with exit code 70 (use -v to see invocation)
clang version 13.0.1
Target: powerpc64le-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/tce/packages/clang/clang-ibm-13.0.1/release/bin
@rchen20
Copy link
Member Author

rchen20 commented May 26, 2022

@trws Does this mean that there are 2 definitions of camp::resources::Host::get_default somewhere, and clang doesn't know which one to pick?

@kab163
Copy link
Contributor

kab163 commented May 26, 2022

More details here: https://lc.llnl.gov/gitlab/radiuss/RAJA/-/pipelines/175291
And PR (draft) #1270

@trws
Copy link
Member

trws commented May 26, 2022

That looks like somehow it is being implicitly declare targeted, and accessed from the GPU-side code in a way that isn't valid. In a way it means there are two of them, sort-of, sometimes, but I would have expected that to fail, or at least give a diagnostic, much earlier.

@trws
Copy link
Member

trws commented May 26, 2022

Translating that, I think camp::resrouces::Host::get_default() is being called inside a device context somewhere. Which isn't really supported. The complete lack of any file and line information is concerning through, not sure where it's happening. Maybe we could get a better diagnostic by explicitly forcing the get_default function to be compiled only for the host, #pragma omp begin declare target device(host) with a corresponding end directive wrapped around it might make it fail earlier when there's line information available.

@trws
Copy link
Member

trws commented May 26, 2022

Also, come to think of it, here's a fantastically inscrutable error message for you @jdoerfert. 😄

@jdoerfert
Copy link

Same underlying reason as in
llvm/llvm-project#54854
and
llvm/llvm-project#48651

Backends (AMD and NVPTX) do not support all atomic instruction + atomic scope combinations.

@rhornung67 rhornung67 added the reviewed Mark with this label when issue has been discussed by team label Jun 7, 2022
@rchen20
Copy link
Member Author

rchen20 commented Nov 15, 2022 •
edited

Hi @jdoerfert @trws, I'm trying to figure out where exactly in RAJA this is tripping up. Is there a way to ask clang/llvm for detailed instruction selection output? I've seen very old versions of clang use this set of options -mllvm -debug-pass=isel, but cannot find the modern equivalent of that.

@rchen20 rchen20 self-assigned this Nov 15, 2022
@jdoerfert
Copy link

This happens still with upstream clang? Can you just do -save-temps and attach the .bc file?

@rchen20 rchen20 linked a pull request Dec 6, 2022 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rchen20 rchen20

Labels
reviewed Mark with this label when issue has been discussed by team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Draft: Clang OpenMP Target Problems LLNL/RAJA
5 participants
@trws @jdoerfert @kab163 @rhornung67 @rchen20