Troubles compiling swift framework due to protobuf errors

[chokokatana]

I had been using v4.0.24 and decided to update my git fork to v4.0.35. After this I'm able to compile and generate android libraries, but I fail to run ./tools/ios-xcframework:

[trezor-crypto] Compiling address.c
[protobuf] Compiling zero_copy_stream_impl_lite.cc
[protobuf] Compiling zero_copy_stream_impl.cc
[protobuf] Compiling zero_copy_stream.cc
[protobuf] Compiling wrappers.pb.cc
[protobuf] Compiling wire_format_lite.cc
❌ …/swift/protobuf/google/protobuf/wire_format_lite.cc:589:8: use of undeclared identifier 'IsStructurallyValidUTF8'
  if (!IsStructurallyValidUTF8(data, size)) {
       ^
[protobuf] Compiling wire_format.cc
[protobuf] Compiling utility.cc
[protobuf] Compiling unknown_field_set.cc
[protobuf] Compiling type_resolver_util.cc
[protobuf] Compiling type_info.cc
[protobuf] Compiling type.pb.cc
[protobuf] Compiling tokenizer.cc
** BUILD DOCUMENTATION FAILED **


The following build commands failed:
	CompileC …/swift/build/docsData/Build/Intermediates.noindex/TrustWalletCore.build/Debug-maccatalyst/protobuf.build/Objects-normal/arm64/wire_format_lite.o …/swift/protobuf/google/protobuf/wire_format_lite.cc normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'protobuf' from project 'TrustWalletCore')
(1 failure)

If I try to act clever and remove the tools/ios-doc step from ./tools/ios-xcframework then I get a different kind of errors:

…
[18:32:43]: ▸ Compiling zero_copy_stream_impl_lite.cc
[18:32:43]: ▸ Compiling zero_copy_stream_impl.cc
[18:32:43]: ▸ Compiling zero_copy_stream.cc
[18:32:43]: ▸ Compiling wrappers.pb.cc
[18:32:43]: ▸ Compiling wire_format_lite.cc
[18:32:43]: ▸ ❌  …/swift/protobuf/google/protobuf/wire_format_lite.cc:589:8: use of undeclared identifier 'IsStructurallyValidUTF8'
[18:32:43]: ▸ if (!IsStructurallyValidUTF8(data, size)) {
[18:32:43]: ▸        ^
[18:32:43]: ▸ Compiling wire_format.cc
[18:32:43]: ▸ Compiling utility.cc
[18:32:43]: ▸ Compiling unknown_field_set.cc
[18:32:43]: ▸ Compiling type_resolver_util.cc
[18:32:43]: ▸ Building library libtrezor-crypto.a
[18:32:43]: ▸ Compiling type_info.cc
[18:32:43]: ▸ Compiling type.pb.cc
[18:32:43]: ▸ Compiling tokenizer.cc
[18:32:43]: ▸ Compiling timestamp.pb.cc
[18:32:43]: ▸ ** ARCHIVE FAILED **
…

It seems that for some reason I've broken my protobuf setup, or am using incompatible versions. Any ideas how could I get to compile the swift framework again?

[satoshiotomakan]

Hi @chokokatana, could you please try to run ./install-dependencies script - it will reinstall Protobuf library with the enabled clang version

[chokokatana]

Oh, forgot about that, thought it would be part of the bootstrap. Still fails with the same error, but somehow it manages to spit one more, because why not:

{ platform:macOS, arch:arm64, variant:Mac Catalyst, id:00006000-0004246C3E62801E, name:My Mac }
{ platform:macOS, arch:x86_64, variant:Mac Catalyst, id:00006000-0004246C3E62801E, name:My Mac }
{ platform:macOS, arch:arm64, variant:Designed for [iPad,iPhone], id:00006000-0004246C3E62801E, name:My Mac }
{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
{ platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }
{ platform:macOS, variant:Mac Catalyst, name:Any Mac }
{ platform:iOS Simulator, id:BEE87AD3-986A-4728-95AB-53C762E7C729, OS:15.2, name:iPhone 13 iOS 15.2 }
{ platform:iOS Simulator, id:BEE87AD3-986A-4728-95AB-53C762E7C729, OS:15.2, name:iPhone 13 iOS 15.2 }
[protobuf] Compiling wire_format_lite.cc
❌ …/swift/protobuf/google/protobuf/wire_format_lite.cc:589:8: use of undeclared identifier 'IsStructurallyValidUTF8'
  if (!IsStructurallyValidUTF8(data, size)) {
       ^
[protobuf] Compiling wrappers.pb.cc
[protobuf] Compiling utility.cc
[protobuf] Compiling wire_format.cc
[protobuf] Compiling unknown_field_set.cc
[protobuf] Compiling type_resolver_util.cc
[protobuf] Compiling type_info.cc
[protobuf] Compiling type.pb.cc
[protobuf] Compiling tokenizer.cc
⚠️  …/swift/protobuf/google/protobuf/io/tokenizer.cc:548:25: 'AddError' is deprecated: Use RecordError [-Wdeprecated-declarations]
      error_collector_->AddError(start_line, start_column,
                        ^
❌ …/swift/protobuf/google/protobuf/io/tokenizer.cc:548:25: 'AddError' is a private member of 'google::protobuf::io::ErrorCollector'
      error_collector_->AddError(start_line, start_column,
                        ^
⚠️  …/swift/protobuf/google/protobuf/io/tokenizer.cc:667:31: 'AddError' is deprecated: Use RecordError [-Wdeprecated-declarations]
            error_collector_->AddError(
                              ^
❌ …/swift/protobuf/google/protobuf/io/tokenizer.cc:667:31: 'AddError' is a private member of 'google::protobuf::io::ErrorCollector'
            error_collector_->AddError(
                              ^
⚠️  …/swift/protobuf/google/protobuf/io/tokenizer.cc:686:29: 'AddError' is deprecated: Use RecordError [-Wdeprecated-declarations]
          error_collector_->AddError(
                            ^
❌ …/swift/protobuf/google/protobuf/io/tokenizer.cc:686:29: 'AddError' is a private member of 'google::protobuf::io::ErrorCollector'
          error_collector_->AddError(
                            ^
[protobuf] Compiling timestamp.pb.cc
** BUILD DOCUMENTATION FAILED **


The following build commands failed:
	CompileC …/swift/build/docsData/Build/Intermediates.noindex/TrustWalletCore.build/Debug-maccatalyst/protobuf.build/Objects-normal/arm64/wire_format_lite.o …/swift/protobuf/google/protobuf/wire_format_lite.cc normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'protobuf' from project 'TrustWalletCore')
	CompileC …/swift/build/docsData/Build/Intermediates.noindex/TrustWalletCore.build/Debug-maccatalyst/protobuf.build/Objects-normal/arm64/tokenizer.o …/swift/protobuf/google/protobuf/io/tokenizer.cc normal arm64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'protobuf' from project 'TrustWalletCore')
(2 failures)

[satoshiotomakan]

Could you share your clang and xcode versions?

[chokokatana]

$ xcodebuild -version
Xcode 15.3
Build version 15E204a

$ clang --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[chokokatana]

Figured out the problem. The issue comes from here due to me not having a standard brew path. When I updated to the latest version I followed my notes but patched the variable HEADER_SEARCH_PATHS in swift/project.yml and swift/common-xcframework.yml. I should have changed SYSTEM_HEADER_SEARCH_PATHS instead. After doing this compilation works again.

Sorry for the trouble, and thanks for your support.

[satoshiotomakan]

Thank you for letting us know! Glad you've managed to resolve the issue