Linux build instructions fail on aarch64 because base library's CPU timept support is implemented only for x64/x86/IA64

[DavidHopkinsFbr]

Hi,

I'm investigating running mini-async-log-c on a Raspberry Pi to log some time-sensitive code. The Raspberry Pi is running the PiCAT 4 version of Raspberry Pi OS, aarch64. This version of Raspberry Pi OS is derived from Debian Buster 10.

I followed the readme's Linux build instructions for Debian/Ubuntu-type systems, but didn't get a successful build. The output from the meson process looks fairly reasonable to my inexperienced eyes:

pi@myhost:~/src/mini-async-log-c $ meson $MYBUILDDIR --buildtype=release
The Meson build system
Version: 1.1.0
Source dir: /home/pi/src/mini-async-log-c
Build dir: /home/pi/src/mini-async-log-c/build
Build type: native build
Project name: mini-async-log-c
Project version: 0.0.1
C compiler for the host machine: cc (gcc 8.3.0 "cc (Debian 8.3.0-6) 8.3.0")
C linker for the host machine: cc ld.bfd 2.31.1
C++ compiler for the host machine: c++ (gcc 8.3.0 "c++ (Debian 8.3.0-6) 8.3.0")
C++ linker for the host machine: c++ ld.bfd 2.31.1
Host machine cpu family: aarch64
Host machine cpu: aarch64

Executing subproject base_library

base_library| Project name: base_library
base_library| Project version: 0.1.0
base_library| C compiler for the host machine: cc (gcc 8.3.0 "cc (Debian 8.3.0-6) 8.3.0")
base_library| C linker for the host machine: cc ld.bfd 2.31.1
base_library| C++ compiler for the host machine: c++ (gcc 8.3.0 "c++ (Debian 8.3.0-6) 8.3.0")
base_library| C++ linker for the host machine: c++ ld.bfd 2.31.1
base_library| Downloading cmocka source from https://cmocka.org/files/1.1/cmocka-1.1.2.tar.xz
Download size: 82300
Downloading: ..........
base_library| Downloading cmocka patch from https://wrapdb.mesonbuild.com/v1/projects/cmocka/1.1.2/1/get_zip
Download size: 4226
Downloading: ..........

Executing subproject base_library:cmocka

cmocka| Project name: cmocka
cmocka| Project version: 1.1.2
cmocka| C compiler for the host machine: cc (gcc 8.3.0 "cc (Debian 8.3.0-6) 8.3.0")
cmocka| C linker for the host machine: cc ld.bfd 2.31.1
cmocka| Compiler for C supports arguments -Wshadow: YES
cmocka| Compiler for C supports arguments -Wmissing-prototypes: YES
cmocka| Compiler for C supports arguments -Wcast-align: YES
cmocka| Compiler for C supports arguments -Werror=address: YES
cmocka| Compiler for C supports arguments -Werror=strict-prototypes: YES
cmocka| Compiler for C supports arguments -Werror=write-strings: YES
cmocka| Compiler for C supports arguments -Werror=implicit-function-declaration: YES
cmocka| Compiler for C supports arguments -Werror=pointer-arith: YES
cmocka| Compiler for C supports arguments -Werror=declaration-after-statement: YES
cmocka| Compiler for C supports arguments -Werror=return-type: YES
cmocka| Compiler for C supports arguments -Werror=uninitialized: YES
cmocka| Compiler for C supports arguments -Wimplicit-fallthrough: YES
cmocka| Compiler for C supports arguments -Werror=strict-overflow: YES
cmocka| Compiler for C supports arguments -Wstrict-overflow=2: YES
cmocka| Compiler for C supports arguments -Wno-format-zero-length: YES
cmocka| Compiler for C supports arguments -Wformat: YES
cmocka| Compiler for C supports arguments -Werror=format-security: NO
cmocka| Compiler for C supports arguments -Wno-gnu-zero-variadic-macro-arguments: NO
cmocka| Compiler for C supports arguments -fno-common: YES
cmocka| Check usable header "assert.h" : YES
cmocka| Check usable header "inttypes.h" : YES
cmocka| Check usable header "io.h" : NO
cmocka| Check usable header "malloc.h" : YES
cmocka| Check usable header "memory.h" : YES
cmocka| Check usable header "setjmp.h" : YES
cmocka| Check usable header "signal.h" : YES
cmocka| Check usable header "stdarg.h" : YES
cmocka| Check usable header "stddef.h" : YES
cmocka| Check usable header "stdint.h" : YES
cmocka| Check usable header "stdio.h" : YES
cmocka| Check usable header "stdlib.h" : YES
cmocka| Check usable header "string.h" : YES
cmocka| Check usable header "strings.h" : YES
cmocka| Check usable header "sys/stat.h" : YES
cmocka| Check usable header "sys/types.h" : YES
cmocka| Check usable header "time.h" : YES
cmocka| Check usable header "unistd.h" : YES
cmocka| Checking whether type "struct timespec" has member "tv_sec" : YES
cmocka| Checking for function "calloc" : YES
cmocka| Checking for function "exit" : YES
cmocka| Checking for function "fprintf" : YES
cmocka| Checking for function "free" : YES
cmocka| Checking for function "longjmp" : YES
cmocka| Checking for function "siglongjmp" : YES
cmocka| Checking for function "malloc" : YES
cmocka| Checking for function "memcpy" : YES
cmocka| Checking for function "memset" : YES
cmocka| Checking for function "printf" : YES
cmocka| Checking for function "setjmp" : YES
cmocka| Checking for function "signal" : YES
cmocka| Checking for function "strsignal" : YES
cmocka| Checking for function "strcmp" : YES
cmocka| Checking for function "clock_gettime" : YES
cmocka| Checking for function "snprintf" : YES
cmocka| Checking for function "vsnprintf" : YES
cmocka| Checking if "Thread Local Storage" compiles: YES
cmocka| Library rt found: YES
cmocka| Header "time.h" has symbol "CLOCK_REALTIME" : YES
cmocka| Checking for size of "void *" : 8
cmocka| Configuring config.h using configuration
cmocka| Build targets in project: 1
cmocka| Subproject cmocka finished.

base_library| Run-time dependency threads found: YES
base_library| Configuring config.h using configuration
base_library| Build targets in project: 17
base_library| Subproject base_library finished.

Dependency threads found: YES unknown (cached)
Configuring config.h using configuration
Build targets in project: 45

mini-async-log-c 0.0.1

  Subprojects
    base_library: YES
    cmocka      : YES

  User defined options
    buildtype   : release

Found ninja-1.10.1 at /usr/bin/ninja
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.

I'm not sure if the "missing usable header" for io.h is a problem. My system does have build-essential installed and I have various files named io.h under /usr/src/linux-headers-.....

Anyway, running the build with ninja $MYBUILDDIR fails, apparently on test code. C is not a language I'm very expert in, but these "implicit declaration of function" warnings look more like something basic is missing from the build environment, to me:

pi@myhost:~/src/mini-async-log-c $ ninja -C $MYBUILDDIR
ninja: Entering directory `build'
[87/155] Compiling C object subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o
In file included from ../subprojects/base_library/test/src/bl/cmocka_pre.h:18,
                 from ../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:3:
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c: In function ‘bl_timept_to_sysclock_diff_test’:
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:16: warning: implicit declaration of function ‘llabs’ [-Wimplicit-function-declaration]
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
                ^~~~~
../subprojects/cmocka-1.1.2/include/cmocka.h:111:28: note: in definition of macro ‘cast_to_largest_integral_type’
     ((LargestIntegralType)(value))
                            ^~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:3: note: in expansion of macro ‘assert_true’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
   ^~~~~~~~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:16: warning: incompatible implicit declaration of built-in function ‘llabs’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
                ^~~~~
../subprojects/cmocka-1.1.2/include/cmocka.h:111:28: note: in definition of macro ‘cast_to_largest_integral_type’
     ((LargestIntegralType)(value))
                            ^~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:3: note: in expansion of macro ‘assert_true’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
   ^~~~~~~~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:16: note: include ‘<stdlib.h>’ or provide a declaration of ‘llabs’
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:23:1:
+#include <stdlib.h>
 /*---------------------------------------------------------------------------*/
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:16:
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
                ^~~~~
../subprojects/cmocka-1.1.2/include/cmocka.h:111:28: note: in definition of macro ‘cast_to_largest_integral_type’
     ((LargestIntegralType)(value))
                            ^~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:41:3: note: in expansion of macro ‘assert_true’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
   ^~~~~~~~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c: In function ‘bl_cpu_timept_vs_timept_test’:
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:52:21: warning: implicit declaration of function ‘bl_cpu_timept_get’; did you mean ‘bl_fast_timept_get’? [-Wimplicit-function-declaration]
   bl_timept cprev = bl_cpu_timept_get();
                     ^~~~~~~~~~~~~~~~~
                     bl_fast_timept_get
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:62:29: warning: implicit declaration of function ‘bl_cpu_timept_to_nsec’; did you mean ‘bl_fast_timept_to_nsec’? [-Wimplicit-function-declaration]
     double ratio = (double) bl_cpu_timept_to_nsec (c - cprev);
                             ^~~~~~~~~~~~~~~~~~~~~
                             bl_fast_timept_to_nsec
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c: In function ‘bl_cpu_timept_to_sysclock_diff_test’:
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:79:23: warning: implicit declaration of function ‘bl_cpu_timept_to_sysclock64_diff_ns’; did you mean ‘bl_cpu_timept_to_sysclock_diff_test’? [-Wimplicit-function-declaration]
   bl_timeoft64 diff = bl_cpu_timept_to_sysclock64_diff_ns();
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                       bl_cpu_timept_to_sysclock_diff_test
In file included from ../subprojects/base_library/test/src/bl/cmocka_pre.h:18,
                 from ../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:3:
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:86:16: warning: incompatible implicit declaration of built-in function ‘llabs’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
                ^~~~~
../subprojects/cmocka-1.1.2/include/cmocka.h:111:28: note: in definition of macro ‘cast_to_largest_integral_type’
     ((LargestIntegralType)(value))
                            ^~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:86:3: note: in expansion of macro ‘assert_true’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
   ^~~~~~~~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:86:16: note: include ‘<stdlib.h>’ or provide a declaration of ‘llabs’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
                ^~~~~
../subprojects/cmocka-1.1.2/include/cmocka.h:111:28: note: in definition of macro ‘cast_to_largest_integral_type’
     ((LargestIntegralType)(value))
                            ^~~~~
../subprojects/base_library/test/src/bl/time_extras/time_extras_test.c:86:3: note: in expansion of macro ‘assert_true’
   assert_true (llabs (diff2 - diff) <= (10 * bl_nsec_in_msec));
   ^~~~~~~~~~~
[89/155] Linking target subprojects/base_library/bl-t_extras-test
FAILED: subprojects/base_library/bl-t_extras-test
cc  -o subprojects/base_library/bl-t_extras-test subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_tests_main.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -Wl,--start-group subprojects/base_library/libbl-base.a subprojects/base_library/libbl-time-extras.a subprojects/cmocka-1.1.2/src/libcmocka.a -Wl,--end-group
/usr/bin/ld: subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o: in function `bl_cpu_timept_to_sysclock_diff_test':
time_extras_test.c:(.text+0x3c): undefined reference to `bl_cpu_timept_to_sysclock64_diff_ns'
/usr/bin/ld: time_extras_test.c:(.text+0xa4): undefined reference to `bl_cpu_timept_to_sysclock64_diff_ns'
/usr/bin/ld: subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o: in function `bl_cpu_timept_vs_timept_test':
time_extras_test.c:(.text+0x128): undefined reference to `bl_cpu_timept_get'
/usr/bin/ld: time_extras_test.c:(.text+0x164): undefined reference to `bl_cpu_timept_get'
/usr/bin/ld: time_extras_test.c:(.text+0x170): undefined reference to `bl_cpu_timept_to_nsec'
collect2: error: ld returned 1 exit status
[91/155] Compiling C++ object subprojects/base_library/bl-nonblo...c-bpm-relacy.p/test_src_bl_mpmc_bpm_relacy_mpmc_bpm_relacy.cpp.o
ninja: build stopped: subcommand failed.

Am I missing some dependency that I should have installed before attempting to build malc?

Re-running the build command generated slightly different output each time, but has eventually stabilised on the following:

pi@myhost:~/src/mini-async-log-c $ ninja -C $MYBUILDDIR
ninja: Entering directory `build'
[1/25] Linking target malc-smoke
FAILED: malc-smoke
cc  -o malc-smoke malc-smoke.p/test_src_malc-smoke_smoke.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -Wl,--start-group libmalc.a subprojects/base_library/libbl-base.a subprojects/base_library/libbl-nonblock.a subprojects/base_library/libbl-time-extras.a subprojects/base_library/libbl-tostr.a subprojects/cmocka-1.1.2/src/libcmocka.a -Wl,--end-group -pthread
/usr/bin/ld: libmalc.a(src_malc_malc.c.o): in function `malc_run_consume_task':
malc.c:(.text+0x6c4): undefined reference to `bl_nsec_to_cpu_timept_max'
/usr/bin/ld: malc.c:(.text+0x6d0): undefined reference to `bl_usec_to_cpu_timept_max'
/usr/bin/ld: malc.c:(.text+0x704): undefined reference to `bl_usec_to_cpu_timept'
/usr/bin/ld: malc.c:(.text+0xa00): undefined reference to `bl_cpu_timept_get'
collect2: error: ld returned 1 exit status
[2/25] Linking target subprojects/base_library/bl-t_extras-test
FAILED: subprojects/base_library/bl-t_extras-test
cc  -o subprojects/base_library/bl-t_extras-test subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_tests_main.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -Wl,--start-group subprojects/base_library/libbl-base.a subprojects/base_library/libbl-time-extras.a subprojects/cmocka-1.1.2/src/libcmocka.a -Wl,--end-group
/usr/bin/ld: subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o: in function `bl_cpu_timept_to_sysclock_diff_test':
time_extras_test.c:(.text+0x3c): undefined reference to `bl_cpu_timept_to_sysclock64_diff_ns'
/usr/bin/ld: time_extras_test.c:(.text+0xa4): undefined reference to `bl_cpu_timept_to_sysclock64_diff_ns'
/usr/bin/ld: subprojects/base_library/bl-t_extras-test.p/test_src_bl_time_extras_time_extras_test.c.o: in function `bl_cpu_timept_vs_timept_test':
time_extras_test.c:(.text+0x128): undefined reference to `bl_cpu_timept_get'
/usr/bin/ld: time_extras_test.c:(.text+0x164): undefined reference to `bl_cpu_timept_get'
/usr/bin/ld: time_extras_test.c:(.text+0x170): undefined reference to `bl_cpu_timept_to_nsec'
collect2: error: ld returned 1 exit status
[3/25] Linking target malc-test
FAILED: malc-test
cc  -o malc-test malc-test.p/test_src_malc_tests_main.c.o malc-test.p/test_src_malc_tls_buffer_test.c.o malc-test.p/test_src_malc_bounded_buffer_test.c.o malc-test.p/test_src_malc_serialization_test.c.o malc-test.p/test_src_malc_entry_parser_test.c.o malc-test.p/test_src_malc_destinations_test.c.o malc-test.p/test_src_malc_array_destination_test.c.o malc-test.p/test_src_malc_file_destination_test.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -Wl,--start-group libmalc.a subprojects/base_library/libbl-base.a subprojects/base_library/libbl-nonblock.a subprojects/base_library/libbl-time-extras.a subprojects/base_library/libbl-tostr.a subprojects/cmocka-1.1.2/src/libcmocka.a -Wl,--end-group -pthread
/usr/bin/ld: libmalc.a(src_malc_destinations_file.c.o): in function `malc_file_dst_open_new_file':
file.c:(.text+0x1c8): undefined reference to `bl_fast_timept_to_sysclock64_diff_ns'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Is there any other information I can provide that would help identify the root cause?

[DavidHopkinsFbr]

Ah, the issue is that libmalc.a needs the following functions and macros in time_extras.h:

• bl_cpu_timept_get()
• bl_cpu_timept_to_sysclock64_diff_ns()
• bl_fast_timept_to_sysclock64_diff_ns()

which are only defined if the symbol BL_HAS_CPU_TIMEPT is true, which is currently only true on x64, x86, and IA64. And the missing helpers from cpu_timept_funcs_arbitrary_base.h are only #included under the same condition. So it looks like aarch64/arm64 is the problem. It would be necessary to replace the x86/x64/IA64 RDTSC intrinsics with ARM equivalents to use this library on any Raspberry Pi.

For reference: https://stackoverflow.com/questions/40454157/is-there-an-equivalent-instruction-to-rdtsc-in-arm

[RafaGago]

Reading BL_HAS_CPU_TIMEPT and failing gracefully (#error) or providing a fallback from malc would probably have been preferable.

You seem to be mostly right. I'm not planning it to add it (and especially testing it) myself but I can take an MR. Note that the heavy lifting is already done by e.g. the Google Benchmark project, it should be pretty straightforward to add:
https://github.com/google/benchmark/blob/main/src/cycleclock.h

On the base library there are these headers which can be useful to make the reduce a bit the #ifdef mess:
https://github.com/RafaGago/base_library/blob/master/include/bl/base/arch.h
https://github.com/RafaGago/base_library/blob/master/include/bl/base/compiler.h

[DavidHopkinsFbr]

That's fair enough. If I end up adding support I will certainly raise a PR for you, but I'm not sure I'll get a chance.