CMake: macOS: glassnt Library Does Not Link

[mfisher31]

Describe the bug
libglassnt.dylib cannot find symbols when linking.

To Reproduce
Steps to reproduce the behavior:

1. Install build dependencies with homebrew
2. Configure cmake with java disabled
3. run make

Expected behavior
Either libraries be built, or cmake configure throw an error when build cannot finish.

Screenshots
N/A

Desktop (please complete the following information):

• OS: macOS 14.4.1 (23E224)
• Versions

% clang --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.4.0

% cmake --version
cmake version 3.29.2

• Project Information:
  • Built from main
  • Git hash: 6a73ca8

Additional context
reported build error:

[ 49%] Building CXX object glass/CMakeFiles/libglassnt.dir/src/libnt/native/cpp/NetworkTablesSettings.cpp.o
[ 49%] Building CXX object glass/CMakeFiles/libglassnt.dir/src/libnt/native/cpp/StandardNetworkTables.cpp.o
[ 49%] Linking CXX shared library ../lib/libglassnt.dylib
Undefined symbols for architecture arm64:
  "_AbslInternalSpinLockWake_lts_20240116", referenced from:
      void absl::lts_20240116::base_internal::CallOnceImpl<void (*)(google::protobuf::FieldDescriptor const*), google::protobuf::FieldDescriptor const*>(std::__1::atomic<unsigned int>*, absl::lts_20240116::base_internal::SchedulingMode, void (*&&)(google::protobuf::FieldDescriptor const*), google::protobuf::FieldDescriptor const*&&) in NetworkTables.cpp.o
  "absl::lts_20240116::base_internal::SpinLockWait(std::__1::atomic<unsigned int>*, int, absl::lts_20240116::base_internal::SpinLockWaitTransition const*, absl::lts_20240116::base_internal::SchedulingMode)", referenced from:
      void absl::lts_20240116::base_internal::CallOnceImpl<void (*)(google::protobuf::FieldDescriptor const*), google::protobuf::FieldDescriptor const*>(std::__1::atomic<unsigned int>*, absl::lts_20240116::base_internal::SchedulingMode, void (*&&)(google::protobuf::FieldDescriptor const*), google::protobuf::FieldDescriptor const*&&) in NetworkTables.cpp.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libglassnt.dylib] Error 1
make[1]: *** [glass/CMakeFiles/libglassnt.dir/all] Error 2
make: *** [all] Error 2

Are these missing symbols something that should be built-in to wpilib or an external dependency?

-- FRC team - 9431

[PeterJohnson]

Those symbols are associated with protobuf. The cmake build pulls in protobuf via wpiutil, and transitive library dependencies should bring it in (libglassnt -> libglass -> wpiutil -> protobuf::libprotobuf). What libprotobuf is cmake picking up? This looks like the same error with another piece of software, but no response yet: BVLC/caffe#7019 (comment)

We also no longer frequently test the "make" build; could you also try the Ninja generator?

[mfisher31]

Trying the Ninja generator now: Protobuf is from homebrew:

-- Found Protobuf: /opt/homebrew/lib/libprotobuf.dylib (found version "5.26.1")

[PeterJohnson]

This also seems similar, so it could be a library link order issue in homebrew packaging of protobuf and/or abseil? bincrafters/community#176

[mfisher31]

I was thinking that too, but order of libraries should only matter when linking against static versions.

Ninja lists these libraries as part of the Linker command. Only one protobuf* library in here.

lib/libntcore.dylib  lib/libglass.dylib  lib/libwpinet.dylib  lib/libwpigui.dylib  lib/libimgui.dylib  lib/libglfw3.a  -framework Cocoa  -framework IOKit  -framework CoreFoundation  lib/libwpimath.dylib  lib/libwpiutil.dylib  /opt/homebrew/lib/libprotobuf.dylib  lib/libfieldImages.dylib

But if I check with pkg-config, not *.cmake packages, several libraries are listed including -labsl_spinlock_wait

% pkg-config --libs protobuf
-L/opt/homebrew/Cellar/protobuf/26.1/lib -L/opt/homebrew/Cellar/abseil/20240116.2/lib 
-L/opt/homebrew/Cellar/protobuf/26.1/lib -L/opt/homebrew/Cellar/abseil/20240116.2/lib 
-lprotobuf -labsl_log_internal_check_op -labsl_leak_check -labsl_die_if_null 
-labsl_log_internal_conditions -labsl_log_internal_message -labsl_examine_stack 
-labsl_log_internal_format -labsl_log_internal_proto -labsl_log_internal_nullguard 
-labsl_log_internal_log_sink_set -labsl_log_sink -labsl_log_entry -labsl_flags_internal 
-labsl_flags_marshalling -labsl_flags_reflection -labsl_flags_private_handle_accessor 
-labsl_flags_commandlineflag -labsl_flags_commandlineflag_internal -labsl_flags_config 
-labsl_flags_program_name -labsl_log_initialize -labsl_log_globals -labsl_vlog_config_internal -labsl_log_internal_fnmatch -labsl_log_internal_globals -labsl_raw_hash_set -labsl_hash 
-labsl_city -labsl_low_level_hash -labsl_hashtablez_sampler -labsl_statusor -labsl_status 
-labsl_cord -labsl_cordz_info -labsl_cord_internal -labsl_cordz_functions 
-labsl_exponential_biased -labsl_cordz_handle -labsl_crc_cord_state -labsl_crc32c 
-labsl_crc_internal -labsl_crc_cpu_detect -labsl_bad_optional_access -labsl_strerror 
-labsl_str_format_internal -labsl_synchronization -labsl_graphcycles_internal 
-labsl_kernel_timeout_internal -labsl_stacktrace -labsl_symbolize -labsl_debugging_internal 
-labsl_demangle_internal -labsl_malloc_internal -labsl_time -labsl_civil_time -labsl_time_zone -labsl_bad_variant_access -lutf8_validity -lutf8_range -labsl_strings -labsl_strings_internal 
-labsl_string_view -labsl_base -labsl_spinlock_wait -labsl_int128 -labsl_throw_delegate 
-labsl_raw_logging_internal -labsl_log_severity

However, this library doesn't get pulled in with protobuf. I tried adding protobuf::libprotobuf to libglassnt's link_libraries, but the same missing symbol errors appear.

Detecting protobuf with pkg-config over cmake packages might be the way to go. I don't have this problem in Linux.

Otool is showing that this homebrewed protobuf has been itself linked to this spinlock thing.

% otool -L /opt/homebrew/lib/libprotobuf.dylib | grep spinlock
	/opt/homebrew/opt/abseil/lib/libabsl_spinlock_wait.2401.0.0.dylib (compatibility version 2401.0.0, current version 0.0.0)

[PeterJohnson]

It definitely sounds like something is broken in homebrew if find_package on protobuf isn’t pulling in what it needs to actually work. We can do a workaround but I would recommend you open an issue there as well.

[mfisher31]

So maybe this is just a newer v.s. older protobuf then, #6610? If that's the case, this #6609 might not be as much of a workaround by, in the cmake files, if/else'ing around the version number where abseil became a dependency.

[mfisher31]

But I guess that still trickles back to protobuf not bringing it's own deps....

[mfisher31]

Upstream issue previously reported:
protocolbuffers/protobuf#12637

[PeterJohnson]

Also protocolbuffers/protobuf#15604 (comment)

[mfisher31]

Fixed with #6609