Attempt to improve CI setup-with-retry time with nix

[purepani]

No description provided.

[github-actions]

Thanks for contributing to openpilot! In order for us to review your PR as quickly as possible, check the following:

• Convert your PR to a draft unless it's ready to review
• Read the contributing docs
• Before marking as "ready for review", ensure:
  • the goal is clearly stated in the description
  • all the tests are passing
  • the change is something we merge
  • include a route or your device' dongle ID if relevant

[BBBmau]

Would love to see if this is possible. I looked into this also but seemed like although nix is great it wouldn't help too much with cutting down the time. I hope I'm wrong though!

[purepani]

Oh, why did it look like it wouldn't save time?
I hadn't used nix in CI before, but from what I have heard about it(and also expect based on how it works), it works pretty well in terms of speed compared to docker. Unless I'm missing something, I fully think that once I get this actually working, it should pretty easily be under 20 seconds.

The real issue is that I dont know whether nix is a technology that commaai would want to use since, while it is very beautiful and useful in some ways, it can be quite frustrating to work with unless you're super into the technology due to

1. Less than stellar documentation. This is a symptom of the fact that the package repository, nixpkgs, is very decentralized and as such, isn't structured super well
2. A lot of tech debt due to being very old. There are some things about nix that you have to deal with.

If I can figure out a way to do this without forcing commaai to have to switch their entire stack to nix, that would be ideal(and what I'm hoping to do with dream2nix).

[BBBmau]

Oh, why did it look like it wouldn't save time? I hadn't used nix in CI before, but from what I have heard about it(and also expect based on how it works), it works pretty well in terms of speed compared to docker. Unless I'm missing something, I fully think that once I get this actually working, it should pretty easily be under 20 seconds.

The real issue is that I dont know whether nix is a technology that commaai would want to use since, while it is very beautiful and useful in some ways, it can be quite frustrating to work with unless you're super into the technology due to

1. Less than stellar documentation. This is a symptom of the fact that the package repository, nixpkgs, is very decentralized and as such, isn't structured super well

2. A lot of tech debt due to being very old. There are some things about nix that you have to deal with.

If I can figure out a way to do this without forcing commaai to have to switch their entire stack to nix, that would be ideal(and what I'm hoping to do with dream2nix).

I had the exact same thought of nix being able to get it under 20s no problem. They would take it as long as it's not overly complex. My attempt was to use https://github.com/cachix/cachix as it supports the use of it in GHA. It seems like nix would be perfect for this but from going through the docs it didn't come across as a solution to solve it since we'd still have to pull the cached binary (assuming we're using cachix) which wouldn't result in much time being saved, the time to pull an image is the bulk of the build when looking at jobs performed.

But keep in mind this is with my knowledge of nix, I stopped investigating due to seeing it as not a solution for <20s. The docs aren't the greatest (and they're aware of that) so I could've totally missed something that would prove me wrong.

I keep intending to get back to investigating at some point but get sidetracked by other opened issues that catch my interest. If you do get it i would love to learn how