Update astc version (#951) errors on linking on a M1 Mac #965

RefuX · 2024-03-08T06:35:24Z

System
OS: macOS 14.4
CPU: Apple M1

Issue
I cloned the repo and on running the command cmake --build build/mac --config Release --target vulkan_samples -j4 the linking failed with the below errors:

[100%] Linking CXX executable bin/Release/arm64/vulkan_samples
ld: warning: ignoring duplicate libraries: '../components/core/lib/Release/arm64/libvkb__core.a', '../components/unix/lib/Release/arm64/libvkb__unix_platform.a'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[23](astcenc_weight_quant_xfer_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[22](astcenc_weight_align.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[21](astcenc_symbolic_physical.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[20](astcenc_quantization.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[19](astcenc_pick_best_endpoint_format.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[18](astcenc_percentile_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[17](astcenc_partition_tables.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[16](astcenc_mathlib_softfloat.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[15](astcenc_mathlib.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[14](astcenc_integer_sequence.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[13](astcenc_image.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[12](astcenc_ideal_endpoints_and_weights.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[11](astcenc_find_best_partitioning.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[10](astcenc_entry.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[9](astcenc_diagnostic_trace.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[8](astcenc_decompress_symbolic.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[7](astcenc_compute_variance.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[6](astcenc_compress_symbolic.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[5](astcenc_color_unquantize.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[4](astcenc_color_quantize.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[3](astcenc_block_sizes.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '../third_party/astc/Source/lib/Release/arm64/libastcdec-native-static.a[2](astcenc_averages_and_directions.cpp.o)': found architecture 'x86_64', required architecture 'arm64'
Undefined symbols for architecture arm64:
  "astcenc_config_init(astcenc_profile, unsigned int, unsigned int, unsigned int, float, unsigned int, astcenc_config*)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_context_free(astcenc_context*)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_context_alloc(astcenc_config const*, unsigned int, astcenc_context**)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
  "astcenc_decompress_image(astcenc_context*, unsigned char const*, unsigned long, astcenc_image*, astcenc_swizzle const*, unsigned int)", referenced from:
      vkb::sg::Astc::decode(vkb::sg::BlockDim, VkExtent3D, unsigned char const*, unsigned int) in libapps.a[114](astc.cpp.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [app/bin/Release/arm64/vulkan_samples] Error 1
make[2]: *** [app/CMakeFiles/vulkan_samples.dir/all] Error 2
make[1]: *** [app/CMakeFiles/vulkan_samples.dir/rule] Error 2
make: *** [vulkan_samples] Error 2

Workaround (for a fresh install)
My workout was to simply checkout the previous commit, thus avoiding the astc update.

git clone https://github.com/KhronosGroup/Vulkan-Samples.git
git checkout 5f86b120c45c4146f78faa98eb4a370f382837f1
git submodule update --init --recursive

The text was updated successfully, but these errors were encountered:

SaschaWillems · 2024-03-08T11:29:23Z

Thanks for bringing this up. We'll try to fix this asap.

jherico · 2024-03-08T20:47:02Z

I don't have an M1 macbook to test on directly. I'll see if there's a cloud provider I can try to use to test, or if I can replicate the problem on a Raspberry Pi or something.

SaschaWillems · 2024-03-08T20:58:01Z

We do have CI for MacOS, but I guess it's x86 only. Maybe there is a way to add MacOS ARM support?

tomadamatkinson · 2024-03-08T21:00:46Z

I have an M1 air I can test on. I think GitHub released Apple M1/2 runners this year too so I can add these to the CI also

SaschaWillems · 2024-03-08T21:02:48Z

We can probably just replace the x86_64 CI builds with ARM.

jherico · 2024-03-08T21:27:42Z

I've started an AWS M2 instance and I'm working the problem. Hopefully I can find a solution that won't require upstream changes.

jherico · 2024-03-08T21:29:31Z

One possibility would be simply use CMake ARCH detection and preprocessor macros to exclude the ASTC encoder on ARM based macs, since they support ASTC natively and don't need to do the decoding.

Honestly that may be the shortest path to resolving this issue.

tomadamatkinson · 2024-03-08T21:36:53Z

Taking a look also. In terms of the CICD I think we just need to add macos-14 and we have Apple Silicon

Heres the docs

jherico · 2024-03-08T21:55:59Z

@tomadamatkinson I've pushed a (likely) fix PR and cherry-picked your CI changes into it. This changed allowed me to build on the EC2 M2 instance after having the same link error described in the issue.

tomadamatkinson · 2024-03-08T22:04:34Z

The commit you took wont work. You may need to remove your current commit and cherry pick again.

As the runners are in beta it looks like they have limited availability. They seem to take a while to provision https://github.com/KhronosGroup/Vulkan-Samples/actions/runs/8209479230?pr=969

Might be worth only running the Apple silicon task on main because of this

Looks like macos-latest will be moved to macos-14 later in the year https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

jherico · 2024-03-08T22:09:06Z

I've removed your CI commit and also added a comment to my change that points to the upstream bug, so the logic can be removed when it's fixed. Cheers.

As the runners are in beta it looks like they have limited availability. They seem to take a while to provision

Yeah, testing this required allocating an AWS dedicated host for at least 24 hours just so I could run an instance. Not a lot of call for cloud based ARM macs I guess, but unfortunately that's the only place where this bug would happen (on investigation I realized I wouldn't have had a problem on any arm64 arch EXCEPT Apple, because this hinges on the CMake OSX_ARCHITECTURES property of targets).

tomadamatkinson · 2024-03-08T22:16:12Z

Sounds good we can merge yours first and I can leave the Arm runner PR open for feedback. The CI looks to be working now but I think there are some considerations around the provisioning aspect

SaschaWillems added build This is relevant to the build system urgent Need to resolve as soon as possible labels Mar 8, 2024

SaschaWillems mentioned this issue Mar 8, 2024

Update ASTC version #951

Merged

12 tasks

jherico mentioned this issue Mar 8, 2024

Fix ASTC link error on Mac arm64 hardware #970

Merged

12 tasks

jherico mentioned this issue Mar 8, 2024

Builds for wrong-architecture on M1 macs ARM-software/astc-encoder#458

Closed

tomadamatkinson linked a pull request Mar 8, 2024 that will close this issue

Add Apple Silicon Build #969

Open

tomadamatkinson closed this as completed in #970 Mar 10, 2024

tomadamatkinson mentioned this issue Mar 10, 2024

Add Apple Silicon Build #969

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update astc version (#951) errors on linking on a M1 Mac #965

Update astc version (#951) errors on linking on a M1 Mac #965

RefuX commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024 •

edited

jherico commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024

jherico commented Mar 8, 2024

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024 •

edited

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

Update astc version (#951) errors on linking on a M1 Mac #965

Update astc version (#951) errors on linking on a M1 Mac #965

Comments

RefuX commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024 • edited

jherico commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024

jherico commented Mar 8, 2024

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024 • edited

jherico commented Mar 8, 2024

tomadamatkinson commented Mar 8, 2024

SaschaWillems commented Mar 8, 2024 •

edited

tomadamatkinson commented Mar 8, 2024 •

edited