CMake integration fails in cross-compilation contexts

[vogr]

Hi,

For some background: I've been trying to cross-compile a program that uses capnproto with Buildroot, but this fails with the following error:

$HOME/buildroot-builddir/host/aarch64-buildroot-linux-gnu/sysroot/usr/bin/capnp: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory

The error itself is not too important (I'm posting it to help people who might encounter the same message find this issue!): the capnp binary that is being executed here is not compiled for my host operating system but for the guest architecture (here arm64). The build process attempts to run it using the binfmt_misc capability, but fails. The actual capnp binary that should be used here, the one compiled by Buildroot for the host architecture, is installed at $HOME/buildroot-builddir/host/bin/capnp, but is not found by the build process.

I think that the problem is not related to Buildroot, but more generally to how Capnproto is integrated in the CMake build in a cross-compilation setting: even when capnproto is compiled with --with-external-capnp, the CapnProtoTargets.cmake file exposes the capnp executable compiled alongside the library instead of the "external" one. For instance on my system:

• ./host/lib/cmake/CapnProto/CapnProtoTargets.cmake sets CapnProto::capnp_tool to ./host/bin/capnp (for x86-64, the "external" capnp for the host).
• ./host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/cmake/CapnProto/CapnProtoTargets.cmake sets CapnProto::capnp_tool to ./host/aarch64-buildroot-linux-gnu/sysroot/usr/bin/capnp (for arm64).
• during cross-compilation, CMake will attempt to find packes in the target system's sysroot, and therefore find_package(CapnProto) will find ./host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/cmake/CapnProto/CapnProtoConfig.cmake (and the corresponding Targets file). This is correct for the libraries (i.e. CMake will try to link to the arm64 libcapnp), but incorrect for the executable (i.e. Cmake will attempt to run the arm64 capnp on an x86-64 machine).

This problem only manifests itself because the CMakeLists.txt of the program I'm trying to compile uses capnp_generate_cpp to generate source files from the capnp definitions. capnp_generate_cpp uses the variable CAPNP_EXECUTABLE (= CapnProto::capnp_tool) to access the generator, but this variable points to the arm64 capnp in my case.

As a workaround, a developer can use find_program to correct the paths of the capnp binaries right after find_program:

# Capnproto
find_package(CapnProto CONFIG REQUIRED)

# Override the capnp paths, as find_package(CapnProto) picks
# the binaries compiled for the target architecture instead of the ones
# for the host
find_program(CAPNP_EXECUTABLE_HOST capnp REQUIRED) 
set_target_properties(CapnProto::capnp_tool PROPERTIES
  IMPORTED_LOCATION ${CAPNP_EXECUTABLE_HOST}
)

find_program(CAPNPC_CXX_EXECUTABLE_HOST capnpc-c++ REQUIRED)
set_target_properties(CapnProto::capnpc_cpp PROPERTIES
  IMPORTED_LOCATION ${CAPNPC_CXX_EXECUTABLE_HOST}
)

find_program(CAPNPC_CAPNP_EXECUTABLE_HOST capnpc-capnp REQUIRED)
set_target_properties(CapnProto::capnpc_capnp PROPERTIES
  IMPORTED_LOCATION ${CAPNPC_CAPNP_EXECUTABLE_HOST}
)

In my opinion, something along thoses lines in the CapnProtoTargets.cmake file would make more sense than the hardcoded paths currently being used: in any cross-compilation scenario, looking for the executable next to the libraries is going to result in paths to unusable programs. However this an imperfect solution for developpers who have multiple versions of capnp installed on their system, as the library (found through find_package() in CMAKE_PREFIX_PATH) and the executable (found through find_program() in the PATH) might not match.

To sum up: I'm not sure how this situation could be made better in the general case, but if anyone stumbles into the same error, at least there's a workaround!

Thank you for all the work,

Valentin

[kentonv]

Thanks for the report. I'm happy to accept a PR that improves how the cmake files handle this situation.

[Sabbirshubho]

I am trying to compile calculator example with yocto(kirkstone version). My host is x86 and target is aarch64. but it fails to link the libraries.

/recipe-sysroot-native/usr/lib/libkj.a(common.c++.o): Relocations in generic ELF (EM: 62)
/recipe-sysroot-native/usr/lib/libkj.a: error adding symbols: file in wrong format
| collect2: error: ld returned 1 exit status

I yocto if I build individually capnp, I see the those static libraries are for aarch64. but if I link this with my custom recipe as a DEPEND, I get above mention error. I checked those static libraries are then build for x86. Any idea how to fix this error?

[Sabbirshubho]

Ok now I see these error if I try to create shared library (with -DBUILD_SHARED_LIBS=ON) for aarch64 architecture from x86 architecture. I am using open embedded recipe for compiling.

/recipe-sysroot-native/usr/lib/libcapnp.a(layout.c++.o): Relocations in generic ELF (EM: 62)

capnproto/1.0.2-r0/recipe-sysroot-native/usr/bin/aarch64-cosy-linux/../../libexec/aarch64-cosy-linux/gcc/aarch64-cosy-linux/11.4.0/ld: capnproto/1.0.2-r0/recipe-sysroot-native/usr/lib/libz.so: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status

recipe-sysroot-native/usr/bin/aarch64-cosy-linux/../../libexec/aarch64-cosy-linux/gcc/aarch64-cosy-linux/11.4.0/ld: /capnproto/1.0.2-r0/recipe-sysroot-native/usr/lib/libssl.so: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status

Is both of the issue related to cross compilation? Is there any work around? @kentonv , @vogr
Thanks in advance

[kentonv]

Sorry but I don't use cmake, I have no idea what to do about this. Someone who knows cmake will have to propose a solution.