Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use nix-store --export/import instead of nix-copy-closure for build-on-target case #63

Closed

Conversation

exarkun
Copy link
Contributor

@exarkun exarkun commented Dec 10, 2021

Fixes #62

I've tested this with a number of deployments for a host that runs GitLab and its dependencies. My latency to the target server is around 100ms and my upstream bandwidth to it is around 20Mbit/sec. I use build_on_target = true. On master@HEAD it takes about 11 minutes to copy all of the derivations for the system for a GitLab upgrade and 3-4 more minutes to finish the deployment. On this branch the 11 minute step becomes a 20-30 second step and the total deployment finishes in 3-4 minutes instead of 14-15 minutes.

However, sometimes the deployment fails during the --export/--import step with an ssh error that I haven't diagnosed yet. Even when I have to restart the deployment because of this the result is still ~10 minutes faster than on master@HEAD (though I would sure like to fix whatever is causing that failure).

@hacklschorsch
Copy link

LGTM / I like this 👍

@exarkun
Copy link
Contributor Author

exarkun commented Mar 9, 2022

I wonder if there's something else I can do to help this get merged.

@Profpatsch
Copy link

Maybe we should migrate this repository to the https://github.com/nix-community organization if it’s unmaintained, then more contributors can be added.

@dpc
Copy link

dpc commented Jan 9, 2023

See #64

@smulikHakipod
Copy link

Hey @adrian-gierakowski !
Now that you are the maintainer, can we merge it?
Thanks!

@adrian-gierakowski
Copy link
Contributor

@smulikHakipod would you be able to add a terraform var to allow user to choose which method of copying should be used, and set the default value to the original implementation. Thanks!

@smulikHakipod
Copy link

Yeah, sure, will do it on Tuesday

@edolstra
Copy link

Note that nix-store --export / --import is pretty deprecated. Its format is undocumented and it will not have a replacement in the new CLI. It's inefficient because it copies all store paths, not just the ones that are missing on the target.

nix-copy-closure is supposed to send all store paths in one stream, see NixOS/nix@fe1f34f. But maybe that's not working for some reason. (Maybe it only works for the ssh-ng store?)

@adrian-gierakowski
Copy link
Contributor

Note that nix-store --export / --import is pretty deprecated. Its format is undocumented and it will not have a replacement in the new CLI. It's inefficient because it copies all store paths, not just the ones that are missing on the target.

nix-copy-closure is supposed to send all store paths in one stream, see NixOS/nix@fe1f34f. But maybe that's not working for some reason. (Maybe it only works for the ssh-ng store?)

Thanks for the info @edolstra!

@smulikHakipod we should probably point out the above in the terraform var description.

@exarkun given the above info, maybe there is a way you could fix your problem without introducing dependency on a deprecated feature?

@exarkun
Copy link
Contributor Author

exarkun commented Jul 17, 2023

Note that nix-store --export / --import is pretty deprecated. Its format is undocumented and it will not have a replacement in the new CLI. It's inefficient because it copies all store paths, not just the ones that are missing on the target.
nix-copy-closure is supposed to send all store paths in one stream, see NixOS/nix@fe1f34f. But maybe that's not working for some reason. (Maybe it only works for the ssh-ng store?)

Thanks for the info @edolstra!

@smulikHakipod we should probably point out the above in the terraform var description.

@exarkun given the above info, maybe there is a way you could fix your problem without introducing dependency on a deprecated feature?

It looks like the performance improving feature was added to a version of nix that came out after this issue was filed? Then maybe nix-copy-closure using a recent version of nix (newer than 2.16.1, maybe?) is all that is necessary to get this performance improvement - and no other changes to terraform-nixos are required.

If that interpretation sounds right, I might be able to do some performance testing with a new version of nix to see if this is the reality or not - though it might be a while before I get around to doing so (but the issue is already years old so that seems fine).

@smulikHakipod
Copy link

I will have a look in nix-copy-closure now

@smulikHakipod
Copy link

smulikHakipod commented Jul 18, 2023

Using nix copy does work great and fast, yet it does not support NixOS 23.05 (as its uses nix 2.13 which is too old) (it works, its justs just slow on old nix versions), its still probably a better solution I guess. I made a PR.

@exarkun
Copy link
Contributor Author

exarkun commented Jun 6, 2024

It looks like #76 should entirely supersede this.

@exarkun exarkun closed this Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Speed up the process of copying drv files to the remote host
7 participants