Fix 'objdump' call not working with Xcode 13.3 command line tools #3254

mikchmie · 2022-02-01T10:11:33Z

Fixes #3253

tmspzz · 2022-02-01T10:16:28Z

@mikchmie thanks! Will this still work with previous versions of Xcode or macOS?

mikchmie · 2022-02-01T10:22:17Z

I'm checking it right now. So far I've checked options' syntax in following versions of `objdump:

Apple LLVM version 13.1.6 (clang-1316.0.19.2) - Xcode 13.3
Apple LLVM version 13.0.0 (clang-1300.0.29.30) - Xcode 13.2.1
Apple LLVM version 12.0.5 (clang-1205.0.22.9) - Command_Line_Tools_macOS_10.12_for_Xcode_8.3.2

and it looks the same (with two dashes --).

@tmspzz What is the minimum version of Xcode that Carthage supports?

mikchmie · 2022-02-01T10:39:41Z

~~Looks like the syntax was changed in LLVM 10 in 2019.~~

~~Last LLVM 9 docs capture:~~
~~https://web.archive.org/web/20190616123225/http://www.llvm.org:80/docs/CommandGuide/llvm-objdump.html~~
~~First LLVM 10 docs capture:~~
~~https://web.archive.org/web/20190812120449/http://llvm.org:80/docs/CommandGuide/llvm-objdump.html~~

mikchmie · 2022-02-01T10:54:19Z

I've also found LLVM's historic documentation and looks like the change actually happened in LLVM 9 (released 19 Sep 2019).
I suppose this is a more reliable way to pinpoint the moment of change, than the WaybackMachnie.

LLVM 8.0.1 docs (-macho):
https://releases.llvm.org/8.0.1/docs/CommandGuide/llvm-objdump.html

LLVM 9.0.0 docs (--macho):
https://releases.llvm.org/9.0.0/docs/CommandGuide/llvm-objdump.html

According to Wikipedia it means that this new syntax would work in Xcode 11.4+.

mikchmie · 2022-02-01T12:10:19Z

@tmspzz If that's necessary I can add a fallback to the chain, e.g.:

return task.launch(standardInput: nil)
    .flatMapError { _ -> SignalProducer<TaskEvent<Data>, TaskError> in
        // Fallback for 'objdump' in LLVM <9.0.0 (Xcode <11.4) which uses `-` as option prefix instead of `--`
        Task("/usr/bin/xcrun", arguments: [
            "objdump",
            "-macho",
            "-private-header",
            "-non-verbose",
            url.resolvingSymlinksInPath().path,
            ]
        ).launch(standardInput: nil)
    }

krzyzanowskim · 2022-02-02T20:00:57Z

I tried this patch with Xcode 13.2.1 and binary xcframework didn't land in ./Carthage/Build/<Platform>/ as per #3253

mikchmie · 2022-02-02T20:33:38Z

@krzyzanowskim As far as I can tell XCFrameworks (*.xcframework) will never land in ./Carthage/Build/<Platform>/ - instead they should be placed in ./Carthage/Build/ - as they usually contain binaries for multiple platforms. Only "basic" frameworks (*.framework) land in ./Carthage/Build/<Platform>/.

krzyzanowskim · 2022-02-02T20:35:25Z

@mikchmie you're right. I just noticed I face a different issue, where the Xcode project that supposes to use .xcframework, links to a non-existing framework from ./Carthage/Build/<Platform>/

mikchmie · 2022-03-09T12:09:58Z

@tmspzz Are there any plans to fix the issue mentioned in this PR (either by merging my PR or some other way)? Xcode 13.3 Release Candidate has already been released, so it won't be long before it's fully released to the public and will give Carthage users trouble.

daumantas-versockas-vinted · 2022-03-23T15:35:22Z

@tmspzz since Xcode 13.3 is released, we are having trouble with it. Are you planning to resolve this issue in the nearest future?

See: Carthage#3254

stale · 2022-04-27T21:23:10Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

mikchmie · 2022-04-28T07:12:48Z

Is the project itself "stale"? Are there any active maintainers who could make Carthage compatible with latest Xcode versions?

CraigSiemens · 2022-04-29T00:01:12Z

We're also waiting for this fix.

Binary dependencies not copied to Carthage/Build with Xcode 13.3.1 #3269

vilu · 2022-04-29T06:26:18Z

Same! Would be happy to help support this.

CraigSiemens · 2022-04-29T17:05:36Z

Our current workaround is to copy over the frameworks ourself. We've already got a script running Carthage so we added this at the end.

# carthage build ...

mkdir -p Carthage/Build/iOS
destination=$(mktemp -d)
tar_path=$(find "$HOME/Library/Caches/org.carthage.CarthageKit/binaries/GoogleMaps" -iname "GoogleMaps*.gz")
tar -xf "$tar_path" -C "$destination"
find "$destination" -d -iname "*.framework" -exec cp -R {} Carthage/Build/iOS \;
rm -r "$destination"

Ro-M · 2022-05-16T11:56:52Z

Source/CarthageKit/MachHeader.swift

 			url.resolvingSymlinksInPath().path,
 			]
 		)

 		return task.launch(standardInput: nil)
+            .flatMapError { _ -> SignalProducer<TaskEvent<Data>, TaskError> in


@mikchmie So to recap my understanding: You are

launching the Task with the new -- prefix option

in case that fails assume that it failed because it still expects the old - prefix

so you "restart" the Task but now with the old implementation

I think we should not just expect that the error is due to the new prefix. How about instead we check the currently installed version of the objdump via objdump --version which outputs something along the line of this for Command Line Tools bundled with Xcode 13.3.1:

Apple LLVM version 13.1.6 (clang-1316.0.21.2.3)

If the version is greater than or equal to 13.1 (or whatever version already works with it) we use the double-dash prefix, if not we use the single-dash prefix. Wdyt?

@Ro-M First of all - thanks for review 🙂
Yes, what you wrote is correct. And I agree that this is not the most elegant solution. But I see 2 risks in your approach:

We might end up with the same problem that we're trying to solve (-version vs --version) - documentation for LLVM 8.0.1 states that -version should be used. However I've tested manually all major releases from 5.0.0 to 9.0.0 and they seem to support --version.

We'll have to keep fingers crossed that the format of this version string will not change. It already looks a lot different for official (non-Apple) LLVM builds:

LLVM (http://llvm.org/): LLVM version 9.0.0 Optimized build. Default target: x86_64-apple-darwin21.4.0 Host CPU: westmere

Maybe it would be better to use double-dash by default and only fallback to single-dash if we detect that the parsed LLVM version is lower than 9.0.0? That way we would prioritize forward compatibility. The version string format is already known for existing versions and if it changes in the future (and parsing will fail), we will use double-dash by default - which, I guess we can assume, will be used by future versions.

@Ro-M I've updated the implementation according to my proposal above. Please let me know what you think about it.

My guess is that they were still backwards compatible? Although I would have expected them to explicitly mention the deprecations but maybe I'm just missing them. And true about the -version vs --version; -version was available without a short-handle and only nowadays you can use either -v or --version.

So with that cleared up I think it would be best to go with your first approach: just try the objdump command and if it doesn't work use the legacy version. Maybe we can wrap this up nicely like you do right now with a name that suggests we are trying a legacy version. I'd also first try the new syntax before falling back to legacy mode (not the other way around).

About how far we may have to look for backwards compatibility: After looking at the documentation you provided (Thanks!) I can see that the --private-header option (the one explicitly for macho!) was first mentioned in 9.0.0 (not mentioned in 8.0.1 and no release inbetween. That one also already has 2 dashes so it should be backwards compatible as well. On a side note: llvm-objdump itself was first mentioned in 8.0.0.

I've digged a bit deeper. The documentation seems to be incomplete for older versions.

LLVM 3.8.0 is the oldest version working with double-dash (which translates to Xcode 7.3).
In other words LLVM 3.7.0 (Xcode 7.2.1) is the newest NOT working version.

However LLVM 3.7.0 doesn't have the -private-header option - only -private-headers (with an 's'), which has a totally different output. LLVM 3.8.0 does have that option. So to me it seems like we don't need any fallback mechanism after all, because our command wouldn't work in 3.7.0 either way.

There are even 2 comments about Xcode 7.x incompatibility in the MachHeader.swift file:
Line 64:

Note: objdump invocation requires flags unimplemented in Xcode 7.X ⋯ as it stands,
only a moreso-alternative codepath invokes this; be cautious to not invert that.

Line 68:

TODO: Potentially, mdfind other non-version-7.X Xcodes that might contain valid objdump.

BTW I've realized that in Apple's builds objdump --version prints "Clang version" instead of "LLVM version" which used to be quite different in the past (e.g. check Xcode 11 in the table - LLVM 8.0.0 vs Clang 11.0.0).

So to sum up - I think I can reset this PR to its initial commit.

However LLVM 3.7.0 doesn't have the -private-header option - only -private-headers (with an 's'), which has a totally different output. LLVM 3.8.0 does have that option. So to me it seems like we don't need any fallback mechanism after all, because our command wouldn't work in 3.7.0 either way.

There is --private-headers (alias -p which is an option you can use for objdumb on its own (similar to --macho) and there is --private-header (NO alias) which is ONLY available as a sub-option for the --macho command (MACH-O ONLY OPTIONS AND COMMANDS)

But the current implementation doesn't use the -private-headers / -p option, so it doesn't matter, does it?

Running the current command (with single-dash syntax) using objdump 3.7.0:

./objdump -macho -private-header -non-verbose ./GooglePlaces

returns an error:

objdump: Unknown command line argument '-private-header'. Try: './objdump -help' objdump: Did you mean '-private-headers'?

So it seems to me that changing single-dash to double-dash syntax will not introduce any regression (because this code doesn't currently work with LLVM < 3.8.0 either - the only difference will be a single character in the error message). Am I wrong?

I was only pointing out the differences in the two private-header options as that seemed to cause confusion on your side (it did initially for me 😄)

Ro-M · 2022-05-20T07:17:03Z

@tmspzz could you please provide feedback here? Is there anything that you would like us to add / change / explain in order to move this forward?

To summarise
The earliest LLVM version Carthage relies on for this particular command requires LLVM 3.8.0 which is already supporting the usage of the double-dash prefix.

In more detail

objdump is not accepting single-dash prefixed parameters starting with LLVM 13.1.6 which is bundled with the Command Line Tools of Xcode 13.3
The new syntax for objdump requires a double-dash prefix for parameters
This causes Carthage to fail silently when trying to check out binary dependencies
The new syntax is already supported since LLVM 3.8.0 bundled with Xcode 7.2.1
LLVM 3.7.0 does not have the -private-header option which is needed here, so LLVM 3.8.0 is the earliest version that this step has to support

mikchmie · 2022-05-20T19:59:41Z

@Ro-M Thanks for the summary 👍
Just 2 tiny corrections:

objdump is not accepting single-dash prefixed parameters starting with LLVM 13.1.6 which is bundled with the Command Line Tools of Xcode 13.3

For the sake of clarity let's stick to one versioning system (the one described as LLVM in table on Wikipedia). 13.1.6 comes from the Clang version string which Apple's builds print as their version. The official LLVM version is 13.0.0.

The new syntax is already supported since LLVM 3.8.0 bundled with Xcode 7.2.1

Actually it's bundled since Xcode 7.3. Xcode 7.2.1 used LLVM 3.7.0

irmakcan · 2022-06-06T22:09:49Z

This change fixes my issue on Xcode 13.4 as well. Hope it gets merged and released soon.

mneuwert · 2022-09-15T09:15:50Z

Hey dudes, did carthage bite the dust? Is somebody planning to merge it? Not usable anymore with Xcode 13.3 - 14. May be also just call otool -h as higher level abstraction?

Ro-M · 2022-10-12T08:50:38Z

@mneuwert I'm not sure about the current plans from the maintainers. As for this PR I would like to move this forward but need direction as to what's missing here. We currently are using a fork of carthage that also includes this change. For the future we are looking into carthage alternatives that have a more active support 😞

mikchmie · 2023-01-04T12:23:13Z

@giginet Could you review this PR?

hujunfeng added a commit to hujunfeng/Carthage that referenced this pull request Apr 11, 2022

Fix 'objdump' call not working with Xcode 13.3 command line tools

d89eed0

See: Carthage#3254

stale bot added the stale label Apr 27, 2022

stale bot removed the stale label Apr 28, 2022

Fix 'objdump' call not working with Xcode 13.3 command line tools

539ce52

mikchmie force-pushed the fix/framework-identification branch from f089f86 to c1f02b4 Compare May 13, 2022 14:50

Ro-M reviewed May 16, 2022

View reviewed changes

mikchmie force-pushed the fix/framework-identification branch from 1712dc2 to 539ce52 Compare May 20, 2022 06:38

Ro-M approved these changes May 23, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 'objdump' call not working with Xcode 13.3 command line tools #3254

Fix 'objdump' call not working with Xcode 13.3 command line tools #3254

mikchmie commented Feb 1, 2022

tmspzz commented Feb 1, 2022

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

krzyzanowskim commented Feb 2, 2022

mikchmie commented Feb 2, 2022

krzyzanowskim commented Feb 2, 2022

mikchmie commented Mar 9, 2022

daumantas-versockas-vinted commented Mar 23, 2022

stale bot commented Apr 27, 2022

mikchmie commented Apr 28, 2022

CraigSiemens commented Apr 29, 2022

vilu commented Apr 29, 2022 •

edited

CraigSiemens commented Apr 29, 2022 •

edited

Ro-M May 16, 2022

mikchmie May 16, 2022

mikchmie May 18, 2022

Ro-M May 19, 2022

mikchmie May 19, 2022

Ro-M May 19, 2022

mikchmie May 19, 2022

Ro-M May 20, 2022

Ro-M commented May 20, 2022 •

edited

mikchmie commented May 20, 2022

irmakcan commented Jun 6, 2022

mneuwert commented Sep 15, 2022

Ro-M commented Oct 12, 2022

mikchmie commented Jan 4, 2023

Fix 'objdump' call not working with Xcode 13.3 command line tools #3254

Are you sure you want to change the base?

Fix 'objdump' call not working with Xcode 13.3 command line tools #3254

Conversation

mikchmie commented Feb 1, 2022

tmspzz commented Feb 1, 2022

mikchmie commented Feb 1, 2022 • edited

mikchmie commented Feb 1, 2022 • edited

mikchmie commented Feb 1, 2022 • edited

mikchmie commented Feb 1, 2022 • edited

krzyzanowskim commented Feb 2, 2022

mikchmie commented Feb 2, 2022

krzyzanowskim commented Feb 2, 2022

mikchmie commented Mar 9, 2022

daumantas-versockas-vinted commented Mar 23, 2022

stale bot commented Apr 27, 2022

mikchmie commented Apr 28, 2022

CraigSiemens commented Apr 29, 2022

vilu commented Apr 29, 2022 • edited

CraigSiemens commented Apr 29, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ro-M commented May 20, 2022 • edited

mikchmie commented May 20, 2022

irmakcan commented Jun 6, 2022

mneuwert commented Sep 15, 2022

Ro-M commented Oct 12, 2022

mikchmie commented Jan 4, 2023

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

mikchmie commented Feb 1, 2022 •

edited

vilu commented Apr 29, 2022 •

edited

CraigSiemens commented Apr 29, 2022 •

edited

Ro-M commented May 20, 2022 •

edited