libc++ unsupported, inconsistent errors are raised

[al42and]

Bug summary

CUDA does not support libc++ on x86 officially. It seems the situation with HIP is the same, although there's no explicit check. And, from the looks of it, generic target does not work all that well either (at lease when AdaptiveCpp and LLVM themselves are compiled with libstdc++). However, the errors produced are super unhelpful except for SMCP with CUDA.

To Reproduce

With a simple file just including #include <sycl/sycl.hpp>:

#include <sycl/sycl.hpp>

int main()
{
    return 0;
}

CUDA multipass does not work, but shows a helpful error:

$ acpp --acpp-targets=cuda:sm_86 --stdlib=libc++ simple.cpp 
acpp warning: No optimization flag was given, optimizations are disabled by default. Performance may be degraded. Compile with e.g. -O2/-O3 to enable optimizations.
clang++: warning: CUDA version is newer than the latest partially supported version 12.1 [-Wunknown-cuda-version]
In file included from <built-in>:1:
In file included from /usr/lib/llvm-17/lib/clang/17/include/__clang_cuda_runtime_wrapper.h:98:
In file included from /usr/local/cuda/include/host_defines.h:60:
/usr/local/cuda/include/crt/host_defines.h:67:2: error: "libc++ is not supported on x86 system"
   67 | #error "libc++ is not supported on x86 system"
      |  ^
1 error generated when compiling for sm_86.

HIP multipass does not work, but shows a confusing error after replacing the __local function name with an address space cast:

$ acpp --acpp-targets=hip:gfx1034 --stdlib=libc++ simple.cpp 
acpp warning: No optimization flag was given, optimizations are disabled by default. Performance may be degraded. Compile with e.g. -O2/-O3 to enable optimizations.
In file included from simple.cpp:1:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/sycl/sycl.hpp:31:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/sycl/../hipSYCL/sycl/sycl.hpp:58:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/sycl/../hipSYCL/sycl/libkernel/accessor.hpp:42:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/hipSYCL/runtime/runtime.hpp:32:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/hipSYCL/runtime/dag_manager.hpp:37:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/hipSYCL/runtime/dag_submitted_ops.hpp:36:
In file included from /home/aland/local/bin/../include/AdaptiveCpp/hipSYCL/runtime/generic/async_worker.hpp:36:
In file included from /usr/lib/llvm-17/bin/../include/c++/v1/queue:272:
/usr/lib/llvm-17/bin/../include/c++/v1/deque:436:67: error: expected ')'
  436 |   static _LIBCPP_HIDE_FROM_ABI __local_iterator __local(_Iterator __iter) { return __iter.__ptr_; }
      |                                                                   ^
/usr/lib/llvm-17/bin/../include/c++/v1/deque:436:56: note: to match this '('
  436 |   static _LIBCPP_HIDE_FROM_ABI __local_iterator __local(_Iterator __iter) { return __iter.__ptr_; }
      |                                                        ^
/usr/lib/llvm-17/bin/../include/c++/v1/deque:436:57: error: redefinition of '_Iterator' as different kind of symbol
  436 |   static _LIBCPP_HIDE_FROM_ABI __local_iterator __local(_Iterator __iter) { return __iter.__ptr_; }
      |                                                         ^
/usr/lib/llvm-17/bin/../include/c++/v1/deque:428:9: note: previous definition is here
  428 |   using _Iterator = __deque_iterator<_ValueType, _Pointer, _Reference, _MapPointer, _DiffType, _BlockSize>;
      |         ^
/usr/lib/llvm-17/bin/../include/c++/v1/deque:436:77: error: expected expression
  436 |   static _LIBCPP_HIDE_FROM_ABI __local_iterator __local(_Iterator __iter) { return __iter.__ptr_; }
      |                                                                             ^
/usr/lib/llvm-17/bin/../include/c++/v1/deque:449:4: error: expected ';' at end of declaration list
  449 |   }
      |    ^
4 errors generated when compiling for gfx1034.

SSCP compiles the code, but the program crashes immediately:

$ acpp --acpp-targets=generic --stdlib=libc++ simple.cpp 
acpp warning: No optimization flag was given, optimizations are disabled by default. Performance may be degraded. Compile with e.g. -O2/-O3 to enable optimizations.
$ ./a.out 
[AdaptiveCpp Error] hcf_cache: Invalid hcf object (missing object id)
Segmentation fault (core dumped)

Expected behavior

Either libc++ is supported (significant effort) or a consistent compile-time error is emitted.

Describe your setup

$ acpp --acpp-\version
acpp [AdaptiveCpp compilation driver], Copyright (C) 2018-2023 Aksel Alpay and the AdaptiveCpp project
  AdaptiveCpp version: 23.10.0+git.c8d5224b.20240212.branch.develop
  Installation root: /home/aland/local
  Plugin LLVM version: 17, can accelerate CPU: True
  Available runtime backends:
     librt-backend-cuda.so
     librt-backend-omp.so
     librt-backend-hip.so
     librt-backend-ocl.so


Full configuration [can be overriden using environment variables or command line arguments]:
    version-major: 23
    version-minor: 10
    version-patch: 0
    version-suffix: +git.c8d5224b.20240212.branch.develop
    plugin-llvm-version-major: 17
    plugin-with-cpu-acceleration: true
    default-clang: /usr/lib/llvm-17/bin/clang++
    default-targets: omp;generic
    default-cpu-cxx: /usr/bin/clang++-17
    default-rocm-path: /opt/rocm
    default-use-bootstrap-mode: false
    default-is-dryrun: false
    default-use-accelerated-cpu: true
    default-clang-include-path: /usr/lib/llvm-17/lib/clang/17/include/..
    default-sequential-link-line: -L/usr/lib/x86_64-linux-gnu -lboost_context -lboost_fiber -Wl,-rpath=/usr/lib/x86_64-linux-gnu
    default-sequential-cxx-flags: -I/usr/include -D_ENABLE_EXTENDED_ALIGNED_STORAGE
    default-omp-link-line: -L/usr/lib/x86_64-linux-gnu -lboost_context -lboost_fiber -Wl,-rpath=/usr/lib/x86_64-linux-gnu -fopenmp
    default-omp-cxx-flags: -I/usr/include -fopenmp -D_ENABLE_EXTENDED_ALIGNED_STORAGE
    default-is-explicit-multipass: false
    default-save-temps: false
    default-rocm-link-line: -Wl,-rpath=$HIPSYCL_ROCM_PATH/lib -Wl,-rpath=$HIPSYCL_ROCM_PATH/hip/lib -L/opt/rocm/lib -L/opt/rocm/hip/lib -lamdhip64
    default-rocm-cxx-flags: -isystem $HIPSYCL_PATH/include/AdaptiveCpp/hipSYCL/std/hiplike -isystem /usr/lib/llvm-17/lib/clang/17/include/.. -U__FLOAT128__ -U__SIZEOF_FLOAT128__ -I$HIPSYCL_ROCM_PATH/include -I$HIPSYCL_ROCM_PATH/include --rocm-device-lib-path=$HIPSYCL_ROCM_PATH/amdgcn/bitcode --rocm-path=$HIPSYCL_ROCM_PATH -fhip-new-launch-api -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -D__HIP_ROCclr__
    default-nvcxx: (unconfigured)
    default-cuda-path: /usr/local/cuda
    default-cuda-link-line: -Wl,-rpath=$HIPSYCL_CUDA_LIB_PATH -L$HIPSYCL_CUDA_LIB_PATH -lcudart
    default-cuda-cxx-flags: -U__FLOAT128__ -U__SIZEOF_FLOAT128__ -isystem $HIPSYCL_PATH/include/AdaptiveCpp/hipSYCL/std/hiplike

Additional context

Not critical. Just making a public note about a confusing behavior.

[illuhad]

I think a prerequisite is that both acpp-compiled code and the AdaptiveCpp runtime definitely need to use the same standard library. The application tries to pass C++ objects between runtime library and itself, so there needs to be ABI compatibility. This is what you have seen with generic target. Once that is done, libc++ should work too, except for stdpar (where it is also documented that currently libstdc++ is needed).

For the other compilation flows, it's a problem in clang CUDA/HIP although we could ensure that better warnings are generated (and document this limitation as you suggest).

[illuhad]

Current documentation has a small note on standard library use under the table here: https://github.com/AdaptiveCpp/AdaptiveCpp/blob/develop/doc/installing.md#advanced-installation