Cargo.nix generation via cargo2nix is incredibly slow

[Ten0]

I've got a 20k lines Cargo.lock and it takes ~1m to generate the corresponding Cargo.nix.
For something that's supposed to be re-generated every time the Cargo.lock is touched that's unacceptably slow.

During that time I can see the cargo2nix process at a high CPU load (~70% on average I think).
This gives me O(n²) vibes, something like, for each dependency we run a cargo API tool that itself does go through the entire Cargo.lock...

• Is there such a thing?
• Can it be fixed?

Thanks,

[psionic-k]

It can be fixed. Open up the cargo source and check the metadata code. It's output contains all of the necessary information. Consider the feature logic busted. It's going to get burned to the ground if a metadata style solution is used. The template stuff will mostly be intact, but with extra support for platform specific dependencies that happen when deps depend on the target, such as with the Bevy crate.

[Ten0]

Thank you for your quick answer. Good to know that it can be fixed.
I'd like to work on this but I'm not sure I understand what you mean.
Is there a comment in the code somewhere that explains the issue and/or what there is to do? Where is "the cargo source's metadata code's output"?
You seem to have plans for a "metadata style solution". Are these documented somewhere?
Thank you. 🙂

[psionic-k]

Plans are in #301 and possibly other linked issues.

Cargo already outputs the correct information in its metadata command. Cargo2nix uses the cargo library and basically can access all the same information that the metadata command outputs via serde. We need nix expressions as the output instead of the JSON. I'm oversimplifying, but that's the gist.

[Ten0]

#301 seems to be a PR about a large change in the nix code, and seems to have no comments on discussion on slowness in the rust code (which it is because of "cargo2nix process at a high CPU load") or plans for using metadata instead.
There is no issue linked to it.
The commit mentions some n^2 but it's in nix code. (It is scary as well, why isn't that implemented as a builtin in nix with a HashMap? But it seems unrelated.)
Are you sure that's the right source?

[psionic-k]

Check issues tagged with high priority. I'm certain everything you're talking about is somewhere in there.