Skip to content

Spack Environments WG

Jump to bottom Edit New page
Todd Gamblin edited this page May 25, 2017 · 7 revisions

Documents

Meetings

Charter

Email from Todd:

As we discussed on the telcon this week, there has been a lot of discussion lately around better support for environments and project development in Spack. As there have been a number of proposals for this, I’d like to get some consensus on the needed features and functionality before we throw three overlapping solutions into Spack. So far, I know of the following environment-related things:

  1. Brett’s Spack views allow you to link related tools into a single prefix.
  2. spack diy lets you build a project from a working directory instead of a fetched tarball.
  3. Elizabeth’s spack setup builds on DIY and allows you to build a project outside Spack in a development environment, potentially with some Spack dependencies. It also adds some support for loading an environment with a project directory. There is a PR for multi-project spack setup
  4. Jim has been working on something called SpackDev for a while now. I think it has some similar goals.
  5. I have talked to Cyrus and some of the LLNL code teams about their development workflows
    • They are interested in something that would allow concurrent development of 3-4 packages at a time (or more)
    • They’d like to be able to manage development on these packages within checked out source directories
    • They would like this to seamlessly integrate with Spack so that it vendors in other dependencies
  6. I personally would like Spack to have something resembling Conda environments or Nix profiles – i.e. something like spack view that allows you to swap between views, and makes Spack act like homebrew when you have a view loaded (i.e., guarnate a consistent install and immediately allow you to use a package, b/c it’s automatically linked into your PATH). Making changes to these environments transactional and having versioned history would be an obvious next step – this is what Nix does.
  7. Matt Krafczyk is doing work at NCSA on reproducibility that aims to allow:
    • Researchers to easily publish reproducible versions of complex projects
    • Consumers of those projects to easily set them up and modify the source
    • I like this a lot because I think containers are great for reproducing things, but they don’t tell you the full story and don’t let you tweak the source.

Currently the efforts for these things are spread out across different projects, but I think they’re all closely related, and I want to put something(s) in Spack that will do this stuff in a consistent way that we can maintain.

So, I would like to get a group together to talk about possibilities. I am contemplating having some type of Spack workshop in the long-term, as I think we have enough people involved to host something like that.

In the short term, I think it would be good to do a few telcons and have people present what they’re working on. Please fill out the doodle poll above and I’ll host the meetings over LLNL Webex once we’ve worked out when folks can participate. I tried to choose times that will work for people in all the US time zones.

If you don’t want/need to be involved, email me or just don’t fill out the poll. Also, feel free to forward this along to others. I may also advertise this on the Spack list in case there are interested folks who don’t come to the telcons, but this group is more than enough to start.

Add a custom footer
Clone this wiki locally