LDC-src Build: C++ ABI broken

[kassane]

Environment Build Test

OS: Alpine latest (docker)
Tools: ldc 1.33 + llvm/clang/libc++ 17 (alpine-pkgs)
Compiler: ldc-master (1.37.0) building
Arch: x86_64

During a few tests to build the musl-toolchain, I got errors when trying to (static) build it with another C++ ABI (non-gnu) libc++. However, when building on Linux systems, only the C++ ABI (GNU) libstdc++ works correctly.

I believe the issue is not related to the Musl target, but just the C++ ABI conflict.
Briefly, I have attempted build a portable ldc-toolchain for use in a legacy distro without interference from glibc version.

I also have questions about the -DLLVM_IS_SHARED=OFF flag and about the parameters passed by llvm-config.
Can you make LLVM-rootdir completely independent of llvm-config ( fulfilling all necessary LLVM dependencies)?

Note: I think in this case the mistake is also because the LLVM available in package is built by libstdc++ not libc++.

Some errors

[ 10%] Built target ldc-prune-cache
ld.lld: error: undefined symbol: llvm::sampleprof::SampleProfileReader::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, llvm::LLVMContext&, llvm::vfs::FileSystem&, llvm::sampleprof::FSDiscriminatorPass, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlapSampleProfile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, llvm::OverlapFuncFilters const&, unsigned long, llvm::raw_fd_ostream&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlapSampleProfile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, llvm::OverlapFuncFilters const&, unsigned long, llvm::raw_fd_ostream&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced 2 more times

ld.lld: error: undefined symbol: llvm::ProfileSummaryBuilder::getHotCountThreshold(std::__1::vector<llvm::ProfileSummaryEntry, std::__1::allocator<llvm::ProfileSummaryEntry>> const&)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlapSampleProfile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, llvm::OverlapFuncFilters const&, unsigned long, llvm::raw_fd_ostream&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlapSampleProfile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, llvm::OverlapFuncFilters const&, unsigned long, llvm::raw_fd_ostream&))

ld.lld: error: undefined symbol: llvm::MemoryBuffer::getFileOrSTDIN(llvm::Twine const&, bool, bool, std::__1::optional<llvm::Align>)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::errorCodeToError(std::__1::error_code)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::ProfileSummaryBuilder::getEntryForPercentile(std::__1::vector<llvm::ProfileSummaryEntry, std::__1::allocator<llvm::ProfileSummaryEntry>> const&, unsigned long)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced 1 more times

ld.lld: error: undefined symbol: vtable for llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(show_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(order_main(int, char const**))
>>> referenced 12 more times
>>> the vtable symbol may be undefined because the class is missing its key function (see https://lld.llvm.org/missingkeyfunction)

ld.lld: error: undefined symbol: llvm::SampleProfileSummaryBuilder::computeSummaryForProfiles(std::__1::unordered_map<llvm::sampleprof::SampleContext, llvm::sampleprof::FunctionSamples, llvm::sampleprof::SampleContext::Hash, std::__1::equal_to<llvm::sampleprof::SampleContext>, std::__1::allocator<std::__1::pair<llvm::sampleprof::SampleContext const, llvm::sampleprof::FunctionSamples>>> const&)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::ProfileSummaryBuilder::getColdCountThreshold(std::__1::vector<llvm::ProfileSummaryEntry, std::__1::allocator<llvm::ProfileSummaryEntry>> const&)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::sampleprof::ProfileConverter::ProfileConverter(std::__1::unordered_map<llvm::sampleprof::SampleContext, llvm::sampleprof::FunctionSamples, llvm::sampleprof::SampleContext::Hash, std::__1::equal_to<llvm::sampleprof::SampleContext>, std::__1::allocator<std::__1::pair<llvm::sampleprof::SampleContext const, llvm::sampleprof::FunctionSamples>>>&)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::sampleprof::DefaultFunctionPruningStrategy::DefaultFunctionPruningStrategy(std::__1::unordered_map<llvm::sampleprof::SampleContext, llvm::sampleprof::FunctionSamples, llvm::sampleprof::SampleContext::Hash, std::__1::equal_to<llvm::sampleprof::SampleContext>, std::__1::allocator<std::__1::pair<llvm::sampleprof::SampleContext const, llvm::sampleprof::FunctionSamples>>>&, unsigned long)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::sampleprof::SampleProfileWriter::writeWithSizeLimitInternal(std::__1::unordered_map<llvm::sampleprof::SampleContext, llvm::sampleprof::FunctionSamples, llvm::sampleprof::SampleContext::Hash, std::__1::equal_to<llvm::sampleprof::SampleContext>, std::__1::allocator<std::__1::pair<llvm::sampleprof::SampleContext const, llvm::sampleprof::FunctionSamples>>>&, unsigned long, llvm::sampleprof::FunctionPruningStrategy*)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))

ld.lld: error: undefined symbol: llvm::toString(llvm::Error)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(exitWithError(llvm::Error, llvm::StringRef))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(handleMergeWriterError(llvm::Error, llvm::StringRef, llvm::StringRef, bool))
>>> referenced 2 more times

ld.lld: error: undefined symbol: llvm::Twine::str() const
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(merge_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlap_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(loadInput(WeightedFile const&, (anonymous namespace)::SymbolRemapper*, llvm::InstrProfCorrelator const*, llvm::StringRef, WriterContext*))
>>> referenced 9 more times

ld.lld: error: undefined symbol: llvm::raw_fd_ostream::raw_fd_ostream(llvm::StringRef, std::__1::error_code&, llvm::sys::fs::OpenFlags)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(show_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(order_main(int, char const**))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlap_main(int, char const**))
>>> referenced 1 more times

ld.lld: error: undefined symbol: llvm::operator<<(llvm::raw_ostream&, std::__1::nullopt_t)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(show_main(int, char const**))

ld.lld: error: undefined symbol: llvm::BalancedPartitioning::run(std::__1::vector<llvm::BPFunctionNode, std::__1::allocator<llvm::BPFunctionNode>>&) const
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(order_main(int, char const**))

ld.lld: error: undefined symbol: llvm::OverlapStats::accumulateCounts(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, bool)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(overlap_main(int, char const**))

ld.lld: error: undefined symbol: llvm::StringError::StringError(llvm::Twine const&, std::__1::error_code)
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(loadInput(WeightedFile const&, (anonymous namespace)::SymbolRemapper*, llvm::InstrProfCorrelator const*, llvm::StringRef, WriterContext*))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(loadInput(WeightedFile const&, (anonymous namespace)::SymbolRemapper*, llvm::InstrProfCorrelator const*, llvm::StringRef, WriterContext*))

ld.lld: error: undefined symbol: vtable for llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::list<llvm::cl::FormattingFlags, llvm::cl::desc>(llvm::cl::FormattingFlags const&, llvm::cl::desc const&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, bool, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::list<char [15], llvm::cl::desc>(char const (&) [15], llvm::cl::desc const&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::opt<char [12], llvm::cl::initializer<char [1]>, llvm::cl::desc>(char const (&) [12], llvm::cl::initializer<char [1]> const&, llvm::cl::desc const&))
>>> referenced 9 more times
>>> the vtable symbol may be undefined because the class is missing its key function (see https://lld.llvm.org/missingkeyfunction)

ld.lld: error: undefined symbol: vtable for llvm::cl::OptionValue<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::opt<char [12], llvm::cl::initializer<char [1]>, llvm::cl::desc>(char const (&) [12], llvm::cl::initializer<char [1]> const&, llvm::cl::desc const&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::opt<char [15], llvm::cl::value_desc, llvm::cl::desc>(char const (&) [15], llvm::cl::value_desc const&, llvm::cl::desc const&))
>>> referenced by llvm-profdata-17.0.cpp
>>>               CMakeFiles/ldc-profdata.dir/ldc-profdata/llvm-profdata-17.0.cpp.o:(llvm::cl::opt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, false, llvm::cl::parser<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::opt<char [7], llvm::cl::value_desc, llvm::cl::initializer<char [2]>, llvm::cl::desc>(char const (&) [7], llvm::cl::value_desc const&, llvm::cl::initializer<char [2]> const&, llvm::cl::desc const&))
>>> referenced 7 more times
>>> the vtable symbol may be undefined because the class is missing its key function (see https://lld.llvm.org/missingkeyfunction)
gmake[2]: *** [tools/CMakeFiles/ldc-profdata.dir/build.make:97: bin/ldc-profdata] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:423: tools/CMakeFiles/ldc-profdata.dir/all] Error 2

Full log:
ldc2_build_clanglibcxx.log
ldc2_build_ziglibcxx.log

Reference

• Implement --cppstdlib #3711

[kinke]

So IIUC, you are hitting build issues with libc++, for the compiler/bundled tools itself. #3711 concerns user programs being able to use core.stdcpp.* for libc++, with a prebuilt druntime expecting libstdc++ on Linux; this doesn't affect the compiler build itself (the compiler itself isn't using core.stdcpp).

I've never tried libc++ on Linux; but if both LLVM and the compiler are built against it, I'd expect it to work. As you're getting linker errors for C++-only tools like ldmd2, it looks like there's a mismatch between LLVM and the C++ parts of LDC. What's your exact CMake command-line?

Btw, why libc++? The official LDC binaries link libstdc++ fully statically:

ldc/.github/workflows/main.yml

Line 33 in 6ede9a4

-DCMAKE_EXE_LINKER_FLAGS=-static-libstdc++

And libstdc++ is apparently an option with musl, as Alpine's ldc package uses it (https://pkgs.alpinelinux.org/package/edge/community/x86_64/ldc).

[kassane]

What's your exact CMake command-line?

cmake "$ROOTDIR/ldc" \
  -DLLVM_ROOT_DIR="$ROOTDIR/out/llvm-cross-$TARGET-$MCPU" \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CROSSCOMPILING=True \
  -DCMAKE_C_COMPILER=clang \
  -DCMAKE_CXX_COMPILER=clang++ \
  -DCMAKE_INSTALL_PREFIX="$ROOTDIR/out/$TARGET-$MCPU" \
  -DCMAKE_PREFIX_PATH="$ROOTDIR/out/$TARGET-$MCPU" \
  -DCMAKE_SYSTEM_NAME="$TARGET_OS_CMAKE" \
  -DLLVM_IS_SHARED=OFF \
  -DD_COMPILER_FLAGS="-gcc=clang -Xcc=-static -Xcc=-stdlib=libc++ -linker=lld"

cmake --build . --target install --parallel 5

I have had problems building using ldc for musl-abi target (mainly cross-compiling) since issue #4520. However, I use this libc for embedded devices and software portability.

Usually I get into the bad habit of linking libstdc++ with glibc. I do not know what the limitations of this libc are when cross compiling. Because there are custom llvm builds that don't use libstdc++ to cross compile (similar to the zig toolchain).
The biggest deadlock for me is this error in switching C++ABI. Also, the dependency on llvm-config makes it difficult to redirect custom llvm root (libs & include only).

[kinke]

The biggest deadlock for me is this bug in switching C++ABI

What bug? AFAICT, you don't pass any -stdlib=libc++ when compiling the LDC C++ parts. From the various logs that you've added, I only see relevant failures when linking C++ executables (ldmd2 and vendored LLVM tools - no D involved anywhere), so if you thought -DD_COMPILER_FLAGS="-gcc=clang -Xcc=-stdlib=libc++ would somehow be enough to switch compiler and tools to libc++, then that's definitely not the case. You'll most likely need to set CMAKE_CXX_FLAGS or so, as you would for any other C++ project.

[kassane]

What bug?

No, error build only!

AFAICT, you don't pass any -stdlib=libc++ when compiling the LDC C++ parts.

Yeah! There's not much choice from what I've seen. Hopefully I can at least build it with musl static.

so if you thought -DD_COMPILER_FLAGS="-gcc=clang -Xcc=-stdlib=libc++"

This flag was needed because the D Compiler failed to follow the default linker. This may be -DLDC_LINK_MANUALLY related.

You'll most likely need to set CMAKE_CXX_FLAGS or so, as you would for any other C++ project.

Ok.