Skip to content

bouthilx/flow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

58 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

https://www.singularity-hub.org/static/img/hosted-singularity--hub-%23e32929.svg

[ebb &] Flow

Experiment pipeline

TODO

Tools

Deploy code to singularity-hub

Include it in .travis.yml files for automatic deployment after successful tests.

flow-deploy ...

Submit jobs on clusters

Using SmartDispatch as a backend

flow-submit [SCHEDULER OPTIONS] [COMMAND] ...

Can be combined with flow-execute

flow-submit [SCHEDULER OPTIONS] flow-execute shub://bouthilx/my-experiment:version [COMMAND] ...

Execute jobs using singularity containers

flow-execute shub://bouthilx/my-experiment:version [COMMAND] ...

Install

TODO

Configuration

In .bashrc or similar,

SINGULARITY_DIR=/some/path/to/singularity
export SREGISTRY_STORAGE=$SINGULARITY_DIR
export SINGULARITY_CACHEDIR=$SINGULARITY_DIR/cache
export SREGISTRY_DATABASE=$SINGULARITY_DIR
export SREGISTRY_NVIDIA_TOKEN=<SECRET>

export DATA_FOLDER=/some/path/to/data

If using mongodb secured with a certificate,

export CERTIFICATE_FOLDER=/some/path/to/certs

TODO

Containers

https://www.singularity-hub.org/static/img/hosted-singularity--hub-%23e32929.svg

Pytorch 0.3.1

About

Experiment pipeline

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published