Cargo.nix Why does it need to live here?

[krisnova]

I understand that Cargo.nix is a file that "pins" the Cargo dependencies to a specific version/release/etc when compiling Aurae for Nix.

My question isn't what the file it is or what the file does... my question is why is it in our repository?

Few Questions

• Is this file something that the project should support?
• Do we intend on supporting Nix compiling Aurae from scratch in favor of downloading a signed artifact?
• Do we intend to keep these operating system specific files in our core repo?
• Should these operating system specific configs be moved to other repos? (For example: a nix repo?)

---

Basically my concern is that I am not sure why the project needs to track 4,500+ lines of .nix data in our core repo? It is polluting our metrics and bloating the repository.

[hbjydev]

Just one question to answer for me :) I don't wanna get involved in "should it be here/do we wanna care" discussions because I've made exactly 1 PR lol.

Do we intend on supporting Nix compiling Aurae from scratch in favor of downloading a signed artifact?

This just won't work if you run NixOS, as Nix poisons linking for stuff it builds in a way that you can only run it with Nix. However, NixOS (the operating system) relies totally on this and doesn't follow the Linux FHS for the most part, so /lib and /lib64, etc. aren't really reliable. That means that anything compiled in a way other than 110% statically is unlikely to run. I've even seen some binaries that are statically-linked not work on NixOS.

Someone can correct me if I'm wrong but that's been my experience.

Source for the LD stuff not working: NixOS/nixpkgs#107375 (comment)
This holds true still.

[marcusramberg]

I was the one who opened the PR to bring it back, but having said that I'm not sure I love the cargo2nix stuff as it seems like you'd need to keep it in sync with the cargo file as aurae changes? The reason I opened it was the flake.nix file still in the repo was referring to it. Having said that I don't love the cargo2nix stuff as it seems like you'd have to keep this file up to date, and like Nova points out it adds a lot of auto generated code to the project. Isn't there a simpler rust builder we could use? I guess the intent when adding this was for actual development of Aurae and not distribution, however I found it easier to just use rustup inside Nixos as well.

So I think either

1. we should remove both the flake.nix/lock and the cargo.nix file again
2. Improve the flake to not use cargo2nix.

[hbjydev]

Using Rust with Nix gets challenging as soon as you start to include system dependencies (a notoriously hard problem with Nix and Rust is crates that depend on OpenSSL, as setting OpenSSL's LD flags are kinda hard. Not an issue here, but it's just an example). So I feel like Nix in this is probably not all that useful, but it does alienate developers who use NixOS on their systems somewhat, because their dev environments won't be particularly frictionless.

I'm not entirely aware of a better system than cargo2nix, but I do wonder if it could be made to work by just including cargo and rustc in the flake instead

[evanrichter]

I've used naersk to build cargo projects with rust, but even that isn't necessary if all you want is a reproducible dev environment. Naersk doesn't commit it's own lock file. I'm not sure what cargo2nix gains by having the lock either, beyond what cargo.lock already provides

You can probably just use flake.nix and it's relatively small auto generated flake.lock. and use oxalica/rust-overlay for the rust toolchain

[krisnova]

so while I appreciate the context, alternative options, opinions, and collective experience of the above comments they aren't answering my question which of course is:

Why is the Cargo.nix file here?

Or more importantly: Why should it remain here?

I merged the file in originally with @edude03 at KubeCon and I guess what I am getting at is that I'm not really seeing any strong cases for keeping it around.

Maybe another way of framing the discussion:

What will we break if we remove the Cargo.nix file and the associated flake things as well? Does anyone intend on using this in production, and if so can someone please tell me why it specifically needs to live in this repository and not another repository? Is there a technical reason it needs to be here specifically?

Should we just move any of the package management style support to the release repository?

[hbjydev]

@krisnova I'm not sure it makes sense in there, as the Cargo.nix file isn't really package management as much as it is a lockfile, like you mentioned before, and there would have to be some weird CI to make that work well. Either way, I believe Cargo2nix works by reading the Cargo.lock and generating the .nix equivalent from it, so it may break Cargo2nix. As far as I can see it's not used in Aurae's CI, but I'm sure there would be people who want to run it in prod with Nix.

[evanrichter]

I will say, even though I haven't contributed to Aurae, that having a flake.nix/lock that some maintainers do use is going to be super inviting for new contributors. Since I won't have to fuss with any development libraries, specific rust toolchains and targets, etc. I'll just have to run nix develop and be ready to go. That applies to any devs using nix, not just NixOS.

Cargo2nix seems like (just one) nix way to get reproducible rust builds. If your project isn't getting benefit from that, I would say remove the cargo.nix because it does skew the metrics visual.

But please don't get rid of all nix files. It really does make sense for flake.nix to live in-repo

[krisnova]

Thank you @evanrichter

having a flake.nix/lock that some maintainers do use is going to be super inviting for new contributors

This is a strong case for keeping the files! Thank you, this is exactly what I was looking for. I personally don't use these files, so I really have no idea why they are there or why the world needs them.

But please don't get rid of all nix files. It really does make sense for flake.nix to live in-repo

Again this is a strong case for keeping them, and I am likely inclined to just go along with keeping the files in the repo.

Concerns

Okay so it sounds like these files are used so that Nix can build the project. Again, I personally don't use nix so I don't really understand why we need these files here. But I trust you all.

So it looks like nix is a package manager. There is a Nix Package Search service that runs and allows folks to find public packages. So (maybe this is wrong?) I am rationalizing nix as an alternative to something like rpm or apt-get.

Basically my concern here is that the project currently only supports the Makefile as a way of building and compiling from source.

There are a number of housekeeping items that we leverage the Makefile for such as:

• Generating the .rs files from the .proto APIs
• Generating the macros
• Ensuring that the rust musl toolchain is installed
• Generating PKI material for development

and so on.

So my concern is that we end up drifting from the supported make targets by using a bespoke package manager such as nix.

So maybe turning the conversation around, I have a few questions:

• Does nix call the Makefile?
• Who is going to maintain these files if the project decides to change the Makefile?
• What happens if nix is unable to build the project? Is that a issue the maintainers need to solve?
• Does nix use Cargo?
• What dependencies can we guarantee are on the system when using nix?

I guess what I am getting at is that these files are very clearly wanted and welcoming to new users, and that makes me want to keep them around. However -- I am not sure on who is going to own the maintenance burden, and I am worried that people will have some incorrect expectations about the project if they try to build the binary using an independent build system that doesn't work the same way the rest of the project works.

My experience maintaining other large open source projects tells me that the moment something drifts from a uniform way of doing something, is the moment the pissed off users and GitHub issues start flocking to the project. I don't want that. I don't want to break people's nix build accidentally without a strong way of determining that using our normal project checks.

[edude03]

Hey as the guy who originally added the nix stuff I feel like I should chime in, though I’m on my phone so I’ll probably have to follow up - So I agree with nova about project drift and using cargo2nix was the wrong choice for this. My original plan was to include an update script but I think the effort outweighs the benefit since someone still has to run the update script and commit the updates. My updated plan (which I don’t mind implementing and maintaining) is to only use nix to provide the devshell, so rustc cargo and protoc. It can call the makefile however in that case we should also have CI to make sure it still works.

…

On Wed, Feb 15, 2023 at 13:20 Kris Nóva ***@***.***> wrote: Thank you @evanrichter <https://github.com/evanrichter> having a flake.nix/lock that some maintainers do use is going to be super inviting for new contributors This is a strong case for keeping the files! Thank you, this is exactly what I was looking for. I personally don't use these files, so I really have no idea why they are there or why the world needs them. But please don't get rid of all nix files. It really does make sense for flake.nix to live in-repo Again this is a strong case for keeping them, and I am likely inclined to just go along with keeping the files in the repo. Concerns Okay so it sounds like these files are used so that Nix <https://nixos.org/> can build the project. Again, I personally don't use nix so I don't really understand why we need these files here. But I trust you all. So it looks like nix is a package manager. There is a Nix Package Search <https://search.nixos.org/packages> service that runs and allows folks to find public packages. So (maybe this is wrong?) I am rationalizing nix as an alternative to something like rpm or apt-get. Basically my concern here is that the project currently only supports the *Makefile* as a way of building and compiling from source. There are a number of housekeeping items that we leverage the *Makefile* for such as: - Generating the .rs files from the .proto APIs - Generating the macros - Ensuring that the rust musl toolchain is installed - Generating PKI material for development and so on. So my concern is that we end up drifting from the supported make targets by using a bespoke package manager such as nix. So maybe turning the conversation around, I have a few questions: - Does nix call the Makefile? - Who is going to maintain these files if the project decides to change the Makefile? - What happens if nix is unable to build the project? Is that a issue the maintainers need to solve? - Does nix use Cargo? - What dependencies can we guarantee are on the system when using nix? I guess what I am getting at is that these files are very clearly wanted and welcoming to new users, and that makes me want to keep them around. However -- I am not sure on who is going to own the maintenance burden, and I am worried that people will have some incorrect expectations about the project if they try to build the binary using an independent build system that doesn't work the same way the rest of the project works. My experience maintaining other large open source projects tells me that the moment something drifts from a uniform way of doing something, is the moment the pissed off users and GitHub issues start flocking to the project. I don't want that. I don't want to break people's nix build accidentally without a strong way of determining that using our normal project checks. — Reply to this email directly, view it on GitHub <#398 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADYXEZBPA2XPHHISBNYAILWXT65DANCNFSM6AAAAAAUZPQ6A4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[krisnova]

Thanks for responding @edude03. Your proposal sounds good to me! Thank you for stepping up and your leadership here. Obviously I will trust whatever you think is best! ❤️

I think all of my concerns are addressed as long as we aren't setting a false expectation that nix will somehow build the software different than the rest of the project I am okay with having the project support Nix 😄

Feel free to close this issue once your suggested changes are merged into main!

Thanks everyone!

[jhillyerd]

@edude03 please take a look at #448 -- I did the minimum to get make auraed-build and make auraescript-build to run.

As noted in the PR, Makefile expects a mutable rustup environment, when I just tried to provide the rust-overlay stable toolchain directly, I got:

$ make auraed-build                                       
rustup target list | grep -qc 'x86_64-unknown-linux-musl (installed)' || \                                                                                                 
rustup target add x86_64-unknown-linux-musl                                                                                                                                
error: no override and no default toolchain set                                                                                                                            
error: no override and no default toolchain set                                      
make: *** [Makefile:134: musl] Error 1    

So I added rustup and the shell hook to set a default toolchain instead.

[jhillyerd]

#464 should close this issue when merged. This could be revisited when Aurae has some stable releases; if it makes sense to run auraed or auraescript on-top of NixOS, a separate repo could be created to contain the flake and lock files. That flake repo could be updated with each release.