Build broken with bevy dependencies

[jrobsonchase]

Minimal crate with dependency: https://github.com/jrobsonchase/hello-rust-nix

It works fine when building directly with cargo, but I get this error when trying to build via nix:

$ nix build -L 2>&1 | sed -n '/> {/s/^ \+> //p' | jq -r .message.rendered
error[E0433]: failed to resolve: could not find `DynamicStruct` in the crate root
  --> src/geometry.rs:29:41
   |
29 | #[derive(Copy, Clone, PartialEq, Debug, Reflect)]
   |                                         ^^^^^^^ not found in the crate root
   |
   = note: this error originates in the derive macro `Reflect` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider importing this struct
   |
1  | use bevy_reflect::DynamicStruct;
   |


error[E0412]: cannot find type `ReflectRef` in the crate root
  --> src/geometry.rs:29:41
   |
29 | #[derive(Copy, Clone, PartialEq, Debug, Reflect)]
   |                                         ^^^^^^^ not found in the crate root
   |
   = note: this error originates in the derive macro `Reflect` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider importing this enum
   |
1  | use bevy_reflect::ReflectRef;
   |


error[E0433]: failed to resolve: could not find `ReflectRef` in the crate root
  --> src/geometry.rs:29:41
   |
29 | #[derive(Copy, Clone, PartialEq, Debug, Reflect)]
   |                                         ^^^^^^^ not found in the crate root
   |
   = note: this error originates in the derive macro `Reflect` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider importing this enum
   |
1  | use bevy_reflect::ReflectRef;
   |


error[E0412]: cannot find type `ReflectMut` in the crate root
  --> src/geometry.rs:29:41
   |
29 | #[derive(Copy, Clone, PartialEq, Debug, Reflect)]
   |                                         ^^^^^^^ not found in the crate root
   |
   = note: this error originates in the derive macro `Reflect` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider importing this enum
   |
1  | use bevy_reflect::ReflectMut;
   |


error[E0433]: failed to resolve: could not find `ReflectMut` in the crate root
  --> src/geometry.rs:29:41
   |
29 | #[derive(Copy, Clone, PartialEq, Debug, Reflect)]
   |                                         ^^^^^^^ not found in the crate root
   |
   = note: this error originates in the derive macro `Reflect` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider importing this enum
   |
1  | use bevy_reflect::ReflectMut;
   |


error: aborting due to 32 previous errors


Some errors have detailed explanations: E0405, E0412, E0432, E0433.

For more information about an error, try `rustc --explain E0405`.

I get the same result when trying to build the bevy_math crate directly on its own (see here), but oddly, it works fine when built in the main repository (fork with nix here). It does not however, work when rebased on the v0.5.0 tag (here), which makes me think it's likely something in one of its dependencies that's getting compiled differently than with cargo.

Additionally, the bevy_dylib crate fails to build in the main repository with a rather long linker error, which I'm attaching separately.

log.txt

[psionic-k]

I reproduced the build failure in your example on the release-0.9.1 branch. I believe the features might not be the same with cargo & nix, so you end up missing a dependency. Do you know which dependency is supposed to be providing Reflect and how it might be toggled with features? There is an outstanding issue of features not matching what results from cargo invocation.

[psionic-k]

Here's bevy 0.7.0 nix build: https://github.com/psionic-k/bevy-build-example

I ran into one hangup with the features I need to check. I had to disable some features on wgpu-hal crate by modifying the Cargo.nix (you just find the features and delete them). Very non-ideal to touch generated code. There's no way to turn these off from rootFeatures AFAIK

• add alsa-sys override to cargo2nix master
• inspect feature logic and figure out why wgpu-hal had all render backends enabled

[Anton-4]

I also ran into this problem with wgpu-hal, I set up a minimal project in case it can help.

[psionic-k]

What I do to fix these is run into main.rs and add some dbg! to show for which feature activation a dx12 feature shows up. Once I know that, I can look at the toml's, straight down the dependencies, and figure out if there's a bug (there's a bug).

Looking at the Cargo.nix's some more, it seems too many features are activated farther down the dependency tree. The last work done was to make features depend correctly on root crate features. As you get deeper in the tree, it's just an enumeration of all the features and no conditions 👎

[Anton-4]

One thing that could be related; for correct feature detection with wgpu Cargo.toml needs to contain resolver = "2" or edition = "2021"(which uses resolver = "2"), see this line of code. I think I read somewhere that cargo2nix builds a synthetic Cargo.toml, perhaps that one does not have resolver = "2" or edition = "2021" set.

[psionic-k]

That's during build phase. When generating the Cargo.nix, we're using cargo 0.61.0 and it can do everything cargo does. Since the extra features show up in Cargo.nix, it's generation that's bugged.

[psionic-k]

cargo tree -e features is helpful to see what the feature expression dependency should look like.

Currently in cargo2nix, the goal was to build with all root crates at least built, meaning any features or dependencies they express in the no default features case will always be on unconditionally.

This behavior involves marking a lot of features downstream as "required", which is whey their expressions have no logic. They are not enabled by anything else, they just are.

However, ignoring the shortcomings of this implementation, crates that are not required in the cargo tree -e features output are being marked as unconditionally on when baked with cargo2nix. This is visible in the Cargo.nix. The wgpu-hal crate has "dx12" on unconditionally.

I need to first convince cargo to generate some expressions where dx12 is not turned on. I can only get it turned on from CLI using --target all, so that's a tell.

This hunk enables dx12 depending on the target
https://github.com/gfx-rs/wgpu/blob/9114283707a8b472412cf4fe685d364327d3a5b4/wgpu-core/Cargo.toml#L69-L70

At least the second cargo invocation is probably wrong. Any feature that depends on a platform, any platform, should not show up in that resolved set and should only show up in the nix expressions guarded by some logic, the platform check and some feature checks.

Platforms and root features form an intersection and the number of platforms is somewhat of an open question... I'm thinking about the approach. What I know for now is that it is very wrong in this important case where dependencies use wildly different backends for different platforms. Graphics is such a thing,

[psionic-k]

Looks like another situation where the model is going to need to be more fully implemented. Expressing the intersection of root features and platforms will be expensive. I don't see a way around it that can be correct.

Might finally be the situation that drives me to look at using guppy to handle the metadata instead of the cargo library.

https://docs.rs/guppy/latest/guppy/graph/struct.EnabledStatus.html

[Anton-4]

Thanks for working on this @psionic-k ❤️

[psionic-k]

You're welcome 🥲 I haven't had any bright ideas. I believe I have to re-architect the expressions and the generator to get reasonable sized expressions that are, most importantly, correct as they can possibly be

[psionic-k]

Updating with information that will later get refined into the readme.

At a high level, cargo2nix is really cargo metadata to Nix. Compared to the JSON output of metadata, we have the advantage of logic expressions. The JSON metadata just prints both versions of a dependency if necessary. We encode variants using logic and using unique attributes like devDependencies.

  "unknown".foo."0.1.0" = overridableMkRustCrate (profileName: rec {
    name = "foo";
    version = "0.1.0";
    registry = "unknown";
    src = fetchCrateLocal workspaceSrc;
    features = builtins.concatLists [
      (lib.optional (rootFeatures' ? "foo/default") "default")
      (lib.optional (rootFeatures' ? "foo/default" || rootFeatures' ? "foo/python") "python")
    ];
    dependencies = {
      ${ if hostPlatform.parsed.cpu.name == "armv7" then "arm" else null } = rustPackages."registry+https://github.com/rust-lang/crates.io-index".arm."0.0.1" { inherit profileName; };
      ${ if hostPlatform.parsed.cpu.name == "x86_64" then "unicode_ident" else null } = rustPackages."registry+https://github.com/rust-lang/crates.io-index".unicode-ident."1.0.1" { inherit profileName; };
      ${ if hostPlatform.parsed.kernel.name == "windows" && hostPlatform.parsed.cpu.name == "aarch64" || hostPlatform.parsed.kernel.name == "windows" && hostPlatform.parsed.cpu.name == "x86_64" then "winapi" else null } = rustPackages."registry+https://github.com/rust-lang/crates.io-index".winapi."0.3.9" { inherit profileName; };
    };
    devDependencies = {
      libc = rustPackages."registry+https://github.com/rust-lang/crates.io-index".libc."0.2.126" { inherit profileName; };
      ${ if hostPlatform.parsed.kernel.name == "linux" then "time_macros" else null } = buildRustPackages."registry+https://github.com/rust-lang/crates.io-index".time-macros."0.2.4" { profileName = "__noProfile"; };
    };
    buildDependencies = {
      spin = buildRustPackages."registry+https://github.com/rust-lang/crates.io-index".spin."0.5.2" { profileName = "__noProfile"; };
    };
  });

At build time, features are enabled to drive conditional compiles using [cfg(feature = "bar")] etc. There is some backend normalization of how the default feature is expressed in mkcrate.nix. The feature consumption in mkcrate.nix has to be right or we get the wrong crate output even if the Cargo.nix is correct.

Varying dependencies, which can happen per-target, can vary features on those dependencies (two variants of the same dependency). Features of course can enable optional deps. Due to all of the reasons why feature logic was changed in resolver=2 or edition=2021, we have to be able to express features per platform, per profile. Currently, dependencies can vary a lot more than features. Features can only change based on rootFeatures, which is improperly implemented still. This breaks platform-specific features / deps such as those in bevy.

The version 0.9.0 logic in main.rs is still interesting. There was a bug with rootFeatures, but it was minor. I'm revisiting this work after finding out more about the results of resolve_ws_with_opts calls.

Cargo's own metadata source shows the correct way in cargo 0.62.0 to get resolve information for all targets and features. Encoding the JSON in Nix requires a bit of plumbing since Nix can express logic but JSON does not.

Reducing cargo metadata for comparison to Cargo.nix

cargo metadata --format-version 1 | jq 'walk(if type == "object" then del(.metadata, .authors, .categories, .description, .license, .license_file, .readme, .repository, .homepage, .keywords, .publish, .manifest_path, .documentation, .targets, .registry, .rust_version, .links, .default_run, .edition, .rename) else . end)
' -C | less

Use --filter-platform aarch64-pc-windows-msvc etc to look at metadata output with different options passed to resolve_ws_with_opts

Cargo's own metadata source is useful for maintaining cargo2nix because changes to cargo that affect the metadata command likely affect how cargo2nix needs to work.

[natto1784]

is there a patchset in work that i can track for this?

[psionic-k]

@natto1784 I'm not tracking burn down toward working on this and there is no branch.

What does exist is design and similar code.

There is a Nix expression generator already in /src. There is code in Cargo metadata from the cargo library that is outputting to JSON what we need instead in Nix expressions.

The work that needs to be done is to create the inputs for metadata's JSON and write Nix expressions instead of JSON. Nix expressions collapse to slightly more compact forms since they can contain logic.

IIRC Bevy specifically exposes some situation where features in dependents need to activate based on the platform or CPU, something I don't think the current expression generator is capable of because it's not supposed to happen - it's an artifact of platform dependencies that have different features enabled when switched on for that platform