Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul Nix Infrastructure #700

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

collinarnett
Copy link

@collinarnett collinarnett commented May 1, 2024

This PR removes the dependecy on haskell.nix and instead relies on the vanilla haskell infrastructure for nix. The primary means of exporting hasktorch will now be an overlay.

Merge Criteria:

  • Support for GPU
  • Support for
    • codegen
    • hasktorch
    • libtorch-ffi
    • libtorch-ffi-helper
    • hasktorch-gradually-typed
  • Update Nix CI
  • Update README with new Nix instructions

@collinarnett
Copy link
Author

Waiting on NixOS/nixpkgs#306749 to be merged for torch 2.3 support.

@junjihashimoto
Copy link
Member

@collinarnett libtorch 2.3 of nixpkgs is available.

@collinarnett
Copy link
Author

OK I'll work on getting this compatible ASAP.

@collinarnett
Copy link
Author

Now all packages build with nix flake check .. I still need to run and package the tests on CPU and CUDA.

@sidkshatriya
Copy link

sidkshatriya commented May 23, 2024

Very fantastic to see everything using vanilla nix. So many lines of custom nix code removed !

I'm eagerly awaiting the merge of this great PR. Looks like its basically complete right ?

P.S. I looked at this in more detail -- this is still missing ability to nix develop and so on

@collinarnett
Copy link
Author

collinarnett commented May 24, 2024

@sidkshatriya

Looks like its basically complete right ?

Yes from a functional perspective it's pretty much finished. You can try it out by using my branch as a flake input and then adding the overlay to your flake.

this is still missing ability to nix develop and so on

At the moment the priority is distributing hasktorch but since this PR uses the default nix haskell infra it's probably safe to assume making a development shell is a simple as using shellFor with the outputs from the overlay.

Either way, I plan on updating https://github.com/collinarnett/hasktorch-skeleton this weekend to give people a demo project of how hasktorch is used as well as packaging the tests to finish up this PR.

@collinarnett
Copy link
Author

@sidkshatriya I've gone ahead and updated hasktorch-skeleton. So go ahead and play around with it and let me know if you have any feedback. It may take a while to compile all the needed packages but once everything is tested a bit more, I'll figure out how to update the CI to get cachix setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants