Skip to content

Commit

Permalink
ci: restore tests in gha, linting, license bump (#70)
Browse files Browse the repository at this point in the history
* ci: restore tests in gha, linting, license bump

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
  • Loading branch information
vsoch committed Mar 28, 2024
1 parent 1960ebc commit 088bd96
Show file tree
Hide file tree
Showing 92 changed files with 1,209 additions and 1,155 deletions.
2 changes: 1 addition & 1 deletion .docs/changelog.md
Original file line number Diff line number Diff line change
@@ -1 +1 @@
../CHANGELOG.md
../CHANGELOG.md
3 changes: 2 additions & 1 deletion .docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
import os
import sys

from recommonmark.parser import CommonMarkParser

sys.path.insert(0, os.path.abspath("."))

# -- General configuration ------------------------------------------------
Expand Down Expand Up @@ -55,7 +57,6 @@
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
from recommonmark.parser import CommonMarkParser

source_parsers = {
".md": CommonMarkParser,
Expand Down
4 changes: 4 additions & 0 deletions .github/dev-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pre-commit
black==23.3.0
isort
flake8
42 changes: 29 additions & 13 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: CI
name: singularity-compose ci

on:
push:
Expand All @@ -8,26 +8,42 @@ on:
branches_ignore: []

jobs:
tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: singularityhub/install-singularity@main
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest semver pytest-runner requests
pip install .[all]
- name: Run unit tests
run: |
which singularity
sudo cp /usr/local/bin/singularity /usr/bin/singularity
pytest -sv scompose/tests/test_depends_on.py
pytest -sv scompose/tests/test_client.py
pytest -sv scompose/tests/test_utils.py
pytest -sv scompose/tests/test_config.py
formatting:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: Setup black environment
run: conda create --quiet --name black

- name: Check formatting with black
run: |
export PATH="/usr/share/miniconda/bin:$PATH"
source activate black
pip install black
black --check scompose
- name: Check Spelling
uses: crate-ci/typos@7ad296c72fa8265059cc03d1eda562fbdfcd6df2 # v1.9.0
with:
files: ./docs/*.md ./docs/*/*.md ./README.md .docs/*.rst .docs/*/*.rst

- name: Check imports with pyflakes
- name: Lint and format Python code
run: |
export PATH="/usr/share/miniconda/bin:$PATH"
source activate black
pip install pyflakes==2.4.0
pyflakes scompose/utils
# Will have some issues
pyflakes scompose/client scompose/project scompose/config || true
pip install -r .github/dev-requirements.txt
pre-commit run --all-files
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
env
.eggs
.docs/_build
dist
Expand Down
32 changes: 32 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
exclude: ".all-contributorsrc"
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-docstring-first
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending

- repo: local
hooks:
- id: black
name: black
language: python
types: [python]
entry: black

- id: isort
name: isort
args: [--filter-files]
language: python
types: [python]
entry: isort

- id: flake8
name: flake8
language: python
types: [python]
entry: flake8
10 changes: 5 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CHANGELOG

This is a manually generated log to track changes to the repository for each release.
Each section should include general headers such as **Implemented enhancements**
This is a manually generated log to track changes to the repository for each release.
Each section should include general headers such as **Implemented enhancements**
and **Merged pull requests**. Critical items to know are:

- renamed commands
Expand All @@ -22,14 +22,14 @@ The versions coincide with releases on pypi.
- fix module import used by check command (0.1.13)
- adding jsonschema validation and check command (0.1.12)
- implement configuration override feature
- implement `--preview` argument for the `check` command
- implement `--preview` argument for the `check` command
- add network->enable option on composer file (0.1.11)
- add network->allocate_ip option on composer file (0.1.10)
- version 2.0 of the spec with added fakeroot network, start, exec, and run options (0.1.0)
- stop option added (equivalent functionality to down)
- stop option added (equivalent functionality to down)
- spython version 0.1.0 with Singularity 3.x or greater required

## [0.0.x](https://github.com/singularityhub/singularity-compose/tree/master) (0.0.x)
## [0.0.x](https://github.com/singularityhub/singularity-compose/tree/master) (0.0.x)
- removed check for sudo when adding network flags (0.0.21)
- singularity-compose down supporting timeout (0.0.20)
- command, ability to associate arguments to the instance's startscript (0.0.19)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![DOI](https://zenodo.org/badge/188852712.svg)](https://zenodo.org/badge/latestdoi/188852712)

This is a simple orchestration library for Singularity containers, akin to
Docker Compose. See the [specification](https://singularityhub.github.io/singularity-compose/#/spec/spec-1.0)
Docker Compose. See the [specification](https://singularityhub.github.io/singularity-compose/#/spec/spec-1.0)
and the [documentation](https://singularityhub.github.io/singularity-compose) for
details, or more examples below.

Expand Down
38 changes: 19 additions & 19 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ or services.

Singularity compose uses Singularity on the backend, so anything that would require sudo (root)
permissions for Singularity is also required for Singularity compose. This includes most
networking commands (e.g., asking to allocate ports) and builds from recipe files.
However, if you are using Singularity v3.3 or higher, you can take advantage of
networking commands (e.g., asking to allocate ports) and builds from recipe files.
However, if you are using Singularity v3.3 or higher, you can take advantage of
[fakeroot](https://sylabs.io/guides/3.3/user-guide/fakeroot.html) to try and get around this.
The snippet below shows how to add fakeroot as an option under a build section:

Expand All @@ -34,15 +34,15 @@ And a complete example is provided [here](https://github.com/singularityhub/sing

### Dependencies

Singularity Compose *must* use a version of [Singularity](https://sylabs.io/guides/latest/user-guide/)
Singularity Compose *must* use a version of [Singularity](https://sylabs.io/guides/latest/user-guide/)
3.2.1 or greater. It's recommended to use the latest (3.3.0 release at the time of this writing) otherwise there was
a bug with some versions of 3.2.1. Singularity 2.x absolutely will not work.
Python 3 is also required, as Python 2 is at end of life.

### singularity-compose.yml

For a singularity-compose project, it's expected to have a `singularity-compose.yml`
in the present working directory. You can look at a simple example here, here is a
in the present working directory. You can look at a simple example here, here is a
version 1.0 spec (before we added networking and exec options):

```yaml
Expand All @@ -60,7 +60,7 @@ instances:
- 80:80
```

and [here](https://github.com/singularityhub/singularity-compose-examples/tree/4241ea8b4e068d93859acb7d2b924702815af0ce/v2.0/ping)
and [here](https://github.com/singularityhub/singularity-compose-examples/tree/4241ea8b4e068d93859acb7d2b924702815af0ce/v2.0/ping)
is a version 2.0 spec that shows adding networking and exec options:

```yaml
Expand All @@ -77,7 +77,7 @@ instances:
options:
- fakeroot
exec:
options:
options:
- "env-file=myvars.env"
command: printenv SUPERHERO
alp2:
Expand All @@ -95,17 +95,17 @@ instances:
- alp1
```

If you are familiar with [docker-compose](https://docs.docker.com/compose/)
the file should look very familiar. A key difference is that instead of
"services" we have "instances." And you guessed correctly - each
section there corresponds to a
If you are familiar with [docker-compose](https://docs.docker.com/compose/)
the file should look very familiar. A key difference is that instead of
"services" we have "instances." And you guessed correctly - each
section there corresponds to a
[Singularity instance](https://sylabs.io/guides/3.2/user-guide/running_services.html)
that will be created. In this guide, we will walk through each of the sections
in detail.

### Instance folders

Generally, each section in the yaml file corresponds with a container instance to be run,
Generally, each section in the yaml file corresponds with a container instance to be run,
and each container instance is matched to a folder in the present working directory.
For example, if I give instruction to build an `nginx` instance from
a `nginx/Singularity.nginx` file, I should have the
Expand Down Expand Up @@ -155,15 +155,15 @@ This will pull a container `nginx.sif` into a `nginx` context folder:
```
It's less likely that you will be able to pull a container that is ready to
go, as typically you will want to customize the
[startscript](https://sylabs.io/guides/3.2/user-guide/definition_files.html#startscript)
go, as typically you will want to customize the
[startscript](https://sylabs.io/guides/3.2/user-guide/definition_files.html#startscript)
for the instance. Now that we understand the basic organization, let's
bring up some instances.
## Quick Start
For this quick start, we are going to use the
[singularity-compose-simple](https://www.github.com/singularityhub/singularity-compose-simple)
For this quick start, we are going to use the
[singularity-compose-simple](https://www.github.com/singularityhub/singularity-compose-simple)
example. Singularity has a networking issue that currently doesn't allow communication
between multiple containers (due to iptables and firewall issues) so for now the most we
can do is show you one container. First, install singularity-compose from pip:
Expand Down Expand Up @@ -255,7 +255,7 @@ $ singularity-compose exec app uname -a
```
When you open your browser to [http://127.0.0.1](http://127.0.0.1)
you should see the upload interface.
you should see the upload interface.
![img/upload.png](img/upload.png)
Expand All @@ -277,7 +277,7 @@ The images that you upload are stored in `images` at the root:

```bash
$ ls images/
2018-02-20-172617.jpg 40-acos.png _upload
2018-02-20-172617.jpg 40-acos.png _upload
```

And static files are in `static`.
Expand Down Expand Up @@ -349,7 +349,7 @@ $ singularity instance start \

Control and customization of these instances is probably the coolest (and not widely
used) feature of Singularity. You can create your own network configurations,
and customie the arguments to the command. Read [here](https://sylabs.io/guides/3.2/user-guide/running_services.html) for more detalis.
and customize the arguments to the command. Read [here](https://sylabs.io/guides/3.2/user-guide/running_services.html) for more detalis.

## Commands

Expand All @@ -360,7 +360,7 @@ Python API, see [here](/singularity-compose/api/).

The [specification](spec/) describes in more detail the sections of the singularity-compose.yml.
For example, in the quick start above, we have a post command for the app instance
that creates a series of folders on the host.
that creates a series of folders on the host.

## Examples

Expand Down
2 changes: 1 addition & 1 deletion docs/api/_sources/changelog.md.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
../CHANGELOG.md
../CHANGELOG.md
2 changes: 1 addition & 1 deletion docs/api/assets/basic.css
Original file line number Diff line number Diff line change
Expand Up @@ -903,4 +903,4 @@ div.math:hover a.headerlink {
#top-link {
display: none;
}
}
}
2 changes: 1 addition & 1 deletion docs/api/assets/css/badge_only.css

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 088bd96

Please sign in to comment.