0.6.3

Changes

• Update Neuroglia-helpers to latest version
• Fix issues in the work directory in kapp
• Improve handling of program files in kbatch

0.6.1

Changes

• Disable the hash check for the docker container download script until we can make a better solution

v0.6.0

• Big improvement to kjupyter

  • After loading up the jupyter server, the program gives an interactive shell within the job
  • When within a kjupyter job, kjupyter subcommands become available:
    • kjupyter log: print the jupyter logs
    • kjupyter console: open up an interactive console attached to an existing active kernel (requires jupyter-console to be installed)
    • kjupyter url: print the browser or server versions of the url
    • kjupyter tunnel: print bash code to set up an ssh tunnel with the server

• Various bug fixes

  • The memory syntax for requesting jobs has been improved
    • units are no longer case-sensitive
    • Allows requests in mebibytes (MiB) and gibibytes (GiB)
  • Fix bug causing gradual degradation of venv/bin/activate files managed by kpy
  • kpy save is now safe from data loss in case of process interruption
  • Major under-the-hood updates to argument parser
  • Improvements to help text and display

0.5.0

New Features

• --venv parameter available in kbatch and krun, allowing preloading of a venv. This is especially useful for scripts.
• kpy export venv <name> --path <path> allows the export of kpy venvs to any path.
• kapp is a new family of commands to download and manage singularity containers. Full details can be found in the docs
  • Containers are pulled by name, but resolved to their content hash. This means that if you, for instance, pull both the :latest image and the specific version :latest points to, no the image won't be pulled twice. On the other hand, if you pull :latest a second time after the container is updated, the newly updated latest image will be pulled.
  • Containers are downloaded and built safely, regardless of size. For large containers, a job is scheduled on an interactive node to build the container.
  • Containers can be aliased, providing easy access to commonly used apps
  • kapp has light wrappers around singularity shell, run, and exec, allowing for easy consumption of kapp managed containers. For complete flexibility, kapp path <container_uri_or_alias> returns the resolved path of the container
  • All containers are automatically aliased in a snakemake readable format. Calling kapp snakemake returns a path you can pass to snakemake via the CLI param --singularity-prefix, allowing snakemake to seamlessly consume any kapp managed container.

Fixes and Improvements

• When kjupyter is called with a --venv provided, it attempts to install jupyter-lab if not already installed
• Escaped quotes in kbatch and krun commands are properly preserved
• kbatch properly runs sbatch scripts: #SBATCH directives will be properly parsed
• The venv prompt name properly updates when calling kpy load. An asterisk (*) appears beside the venv name when modified (e.g. a new package is installed)
• Better error messages

Changes

• Print scheduled time for batch @pvandyken (#33)
• kapp @pvandyken (#35)

🐛 Bug Fixes

• Fix kbatch treatment of scripts and in-job krun @pvandyken (#34)
• Improve quoting of commands @pvandyken (#32)
• Jupyter autoinstall @pvandyken (#31)

0.4.0

Changes

kjupyter:

• Integrates with kpy, so you can directly fire up a jupyter notebook using a kpy saved venv
• When started, it outputs connection links for browser and/or VScode, as well as a copy/paste bash line to open a ssh tunnel, should you need one.

kpy:

• The api is changed so that kpy list lists all the saved venvs. kpy rm is added to delete a saved venv.
• kpy activate, when called without arguments, lists all the created and loaded venvs available to activate (formerly performed by kpy list
• kpy create will try to detect what version of python is currently activated and use that to initialize a new venv, assuming the user didn't specify a version (previously, it would always install using the python version used to install kslurm)

config:

• account can now be set interactively using kpy config account -i. This will start a program checking what accounts are available, along with their 'LevelFS' and displaying a selection menu.
• Separate accounts can be specified for cpu and gpu jobs.

Neuroglia-helpers:

• kslurm now packages drop-in support for neuroglia-helpers. Running kslurm neuroglia-helpers >> $HOME/.bash_profile will add a line sourcing the appropriate file into the .bash_profile, and will recreate nearly all the features enabled through a regular neuroglia-helper install. The profile is set through kpy config neuroglia_profile <profile_name>. Full documentation can be found here

🚀 Features

• Interactive setting of account variable @pvandyken (#29)
• Neuroglia helpers @pvandyken (#28)
• Jupyter kpy integration @pvandyken (#27)
• Update kpy api @pvandyken (#25)
• Typer interface and automatic python version detection @pvandyken (#26)

🐛 Bug Fixes

• Kpy env switch @pvandyken (#24)

🧰 Maintenance

• Updated arg model @pvandyken (#23)
• Add automatic version bump, deployment, and logs @pvandyken (#22)

📝 Documentation

• More complete installation instructions @pvandyken (#30)

v0.3.5

What's Changed

• Include job templates in distribution by @pvandyken in #21

v0.3.4

New kpy command for managing ephemeral python virtualenvs on compute nodes. See the README for details.

Initial release

First release stable enough for public use.