cargoExtraArgs prevents sharing of cargoArtifacts?

[mightyiam]

Thank you for crane!

We noticed (and it is also mentioned at the bottom here) that for a "cargoArtifacts" to be useful it needs to share the same (or similar enough) cargoExtraArgs with its consumer.

That seems to pose a limitation on how useful artifact sharing can be in workspaces with derivations for individual crates.

For example, a workspace with crates foo and bar that are bin crates, where they share dependencies, and there's an individual derivation for each. The call for building foo would have cargoExtraArgs = "-p foo"; and for bar accordingly.

I'm not certain, but I would imagine that optimally, the same cargoArtifacts would be provided to both of them. That buildDepsOnly call would have default cargoExtraArgs, so that it would have all of the dependencies of the workspace. But, according to documentation and our experience, that's not possible. The difference in cargoExtraArgs between the buildDepsOnly call and the buildPackage call results in somehow invalid cache.

It seems that those foo and bar derivations would not share any cache. And that any dependencies they share would be compiled twice. And if we have another binary crate, then thrice. And if we have a workspace wide cargoClippy call, then four times.

Is there any way we could optimize this, please?

[dpc]

This is a standard cargo behavior and tomes have been written about this problem. You'll see the same issue in a local shell when doing cargo build -p <local-crate> vs just cargo build, causing rebuilds etc.

Check through #341 and google around. I don't think there's anything for crane to do here. Would be great if cargo supported some flags that say "always use the whole workspace dependency set".

[mightyiam]

Thank you.

[ipetkov]

To add a bit of extra context here: the issue likely has to do with how cargo handles "feature unification". If foo depends on blah features=a and bar depends on blah features=b, then cargo build would end up building foo, bar, blah features=a,b.

My general advice is to always have the same set of arguments that go into buildDepsOnly and anything else after it to avoid surprises.

Otherwise you may need to build each derivation with multiple invocations to have everything cached. For example:

1. tune buildDepsOnly to call cargo build -p foo && cargo build -p bar (or use something like cargo-hack if you are so inclined
2. then reuse that result across the package-specific derivations

Hope that helps!