Speeding up `carthage outdated` build phases #2698

ctxppc · 2019-02-04T13:56:45Z

Hi everybody,

This is my first issue on this repository and I'm not familiar with the Carthage codebase (yet) to make a PR. I use Carthage to manage my apps' dependencies, and I also use the outdated subcommand as part of the build process (as an Xcode build phase, with --xcode-warnings) to be notified of updates to dependencies. Both updating and checking for updates work as intended. However, in my quest for ever tighter debug turnaround times, I'm trying to decrease my debug build times as much as possible. And one part of the build turnaround is 1 to 3 seconds spent by Carthage to fetch my dependencies' repositories. It doesn't seem Xcode can parallelise this so it's a blocking second or three, on a fast connection.

Unless I'm missing a command line option or implementation detail, I'm proposing that carthage outdated should cache its result (basically skip fetching if it has been done in the past few minutes or hours), possibly as an opt-in (or opt-out) option. It should consistently report outdated repositories regardless of whether it actually fetched anything in this run. I don't think carthage update should do any such caching since it's expected that it fetches all updates that are published as of right now.

Any thoughts on this? Thanks for all the work! 😄

Configuration

carthage install method: [ ] .pkg, [X] homebrew, [ ] source
which carthage: /usr/local/bin/carthage
carthage version: 0.31.2
xcodebuild -version: 10.1 (10B61)
Are you using --no-build? No.
Are you using --no-use-binaries? No.
Are you using --use-submodules? No.
Are you using --cache-builds? Yes.
Are you using --new-resolver? No.

Cartfile

github "ReactiveX/RxSwift" ~> 4.0
github "roberthein/TinyConstraints"

Carthage Output
(Probably) N/A — updating works as intended

Actual outcome
Carthage fetches all dependencies' repositories and build phase takes 1 to 3 seconds to finish, regardless of previous builds preceding it.

Expected outcome
Carthage should skip fetching repositories in the second build soon after the first and the build phase should complete within 0.1 second.

The text was updated successfully, but these errors were encountered:

tmspzz · 2019-02-04T14:50:42Z

Thanks for opening an issue 🎉

Is there a reason why you think carthage should do this as opposed of wrapping carthage commands in a script of your making that does exactly what you propose?

ctxppc · 2019-02-04T15:24:23Z

The README file recommends with a nice one-liner to add this command as a build phase, which is good because without any warning some dependencies might become really outdated without anyone noticing. Given that the subcommand has an --xcode-warnings option, it does seem that it really is intended to be added as a build phase. I'd myself recommend anybody to add this as a build phase. It would therefore make sense to optimise for this use case.

I have a relatively fast wired connection on my desktop computer and currently use only 2 dependencies in this project, yet it takes between 1 and 3 seconds on every single build, and Xcode doesn't do anything in the meantime. I might do hundreds of debug builds every day so that's hundreds of seconds lost waiting for a redundant roundtrip to the network. On a slower connection and with 5 dependencies this can add up to thousands of seconds. Incremental (iterative) debug builds are relatively fast so I can't go fetch a coffee while waiting so it would be great if that build time were as minimal as possible without having to sacrifice a periodical updates check.

Requiring users to write their own script, however, has three big drawbacks:

They have to write more shell code than just a concise Carthage one-liner.
They have to write code that determines when to invoke the command, e.g., by storing the fetch date in a file and only running after some time elapses or by using a random number generator and running with an appropriate probability.
The command with the correct option outputs warnings in Xcode. If the command isn't run (because the user script decided so), no warnings are output and Xcode reports zero warnings. It's easy to miss the warning the one time it is printed between builds. The user script could cache the warnings and print them whenever the Carthage command isn't run but that just means even more bookkeeping code.

So basically it's a lot of work to save a lot of time for a feature I'd recommend everybody to use. It's an (performance) enhancement request so nothing critical. ;-)

tmspzz · 2019-02-04T15:42:01Z

I see your points, but this in my opinion this is a workflow related choice.

Therefore I am reluctant to add this directly in Carthage but, if you want to share your workflow other users could benefit from your contributions here https://github.com/Carthage/workflows

Maybe @Carthage/carthage has other opinions.

mdiep · 2019-02-11T12:33:06Z

I'd love for carthage outdated to be faster. And I think it'd be fine to not always fetch for that command. Maybe it'd be better to add an option that specifies the fetch interval for it? It'll all depend on what the patch looks like.

I agree this repo is not the right place to include a shell script.

mollyIV · 2019-02-24T19:37:24Z

Hello guys 👋 I like this idea a lot and would like to help working on it 🙌To move the discussion a bit forward I created a draft pull request where I started the implementation and asked a few follow-up questions.

Hopefully you don't mind and I would like to kindly ask you to have a look at the draft 👀 😊🙇🏻‍♂️

stale · 2019-03-26T19:40:29Z