SingularityCE 3.9.5

SingularityCE 3.9.5 is a bugfix release, with the following changes:

Changed defaults / behaviours

• make install now installs man pages. A separate make man is not required.

Bug fixes

• GitHub .deb packages correctly include man pages.
• Update dependency to correctly unset variables in container startup environment processing. Fixes regression in v3.9.2 affecting precedence of host/container environment variables.
• Remove subshell overhead when processing large environments on container startup. Reduces container startup time by >25x for a 5000 variable, 500KiB environment.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Source Code

Please use the singularity-ce-3.9.5.tar.gz download below to obtain and install SingularityCE 3.9.5. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

Packages

RPM / DEB packages are provided for:

• Ubuntu 18.04 (bionic)
• Ubuntu 20.04 (focal)
• RHEL/CentOS 7 (el7)
• RHEL/CentOS/Alma/Rocky 8 (el8)

These packages were built with Go 1.17.6

SingularityCE 3.9.4

Bug fixes

• Address timeout in library pull single stream download.

This release includes a single bugfix to address context timeout errors that may be experienced when pulling larger images, or small images over a slow connection, from the recently updated Sylabs Cloud or Singularity Enterprise 2.x Library.

The errors affect SingularityCE >=3.9.0

If you pull images from the Sylabs Cloud or Singularity Enterprise 2.x Library, the new 3.9.4 release provides an immediate fix for the problem.

Separately, Sylabs is working to design and implement a server-side mitigation for the issue. If it is possible, this would mitigate the issue without the need to upgrade your SingularityCE installation.

We sincerely apologize for the recent inconvenience following updates to the Sylabs Cloud. We are working to resolve all remaining issues, and will conduct a review to identify process improvements for the future.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Source Code

Please use the singularity-ce-3.9.4.tar.gz download below to obtain and install SingularityCE 3.9.4. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

Packages

RPM / DEB packages are provided for:

• Ubuntu 18.04 (bionic)
• Ubuntu 20.04 (focal)
• RHEL/CentOS 7 (el7)
• RHEL/CentOS/Alma/Rocky 8 (el8)

These packages were built with Go 1.17.6

SingularityCE 3.9.3

Bug fixes

• Ensure MIGs are visible with --nvccli in non-contained mode, to match the legacy GPU binding behaviour.
• Avoid fd leak in loop device transient error path.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Source Code

Please use the singularity-ce-3.9.3.tar.gz download below to obtain and install SingularityCE 3.9.3. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

Packages

RPM / DEB packages are provided for:

• Ubuntu 18.04 (bionic)
• Ubuntu 20.04 (focal)
• RHEL/CentOS 7 (el7)
• RHEL/CentOS/Alma/Rocky 8 (el8)

These packages were built with Go 1.17.5

SingularityCE 3.9.2

Bug fixes

• Ensure gengodep in build uses vendor dir when present.
• Fix source of a script on PATH and scoping of environment variables in definition files (via dependency update).
• Ensure a local build does not fail unnecessarily if a keyserver config cannot be retrieved from the remote endpoint.
• Correct documentation for sign command r.e. source of key index.
• Restructure loop device discovery to address an issue where a transient EBUSY error could lead to failure under Arvados. Also greedily try for a working loop device, rather than perform delayed retries on encountering EAGAIN, since we hold an exclusive lock which can block other processes.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Source Code

Please use the singularity-ce-3.9.2.tar.gz download below to obtain and install SingularityCE 3.9.2. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

Packages

RPM / DEB packages are provided for:

• Ubuntu 18.04 (bionic)
• Ubuntu 20.04 (focal)
• RHEL/CentOS 7 (el7)
• RHEL/CentOS/Alma/Rocky 8 (el8)

These packages were built with Go 1.17.5

SingularityCE 3.9.1

This is a security release for SingularityCE 3.9, addressing a security issue in SingularityCE's dependencies.

Security Related Fixes

• CVE-2021-41190 / GHSA-77vh-xpmg-72qh: OCI specifications allow ambiguous documents that contain both "manifests" and "layers" fields. Interpretation depends on the presence / value of a Content-Type header. SingularityCE dependencies handling the retrieval of OCI images have been updated to versions that reject ambiguous documents.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Source Code

Please use the singularity-ce-3.9.1.tar.gz download below to obtain and install SingularityCE 3.9.0. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

Packages

⚠️ These packages were built with a Go version (1.17.3) vulnerable to CVE-2021-44717. This is a Go issue, rather than a problem in the SingularityCE code. No direct exploit for SingularityCE has been identified at this time, however ForkExec calls are performed for multiple tasks, and users are encouraged to use updated packages.

RPM / DEB packages are provided for:

• Ubuntu 18.04 (bionic)
• Ubuntu 20.04 (focal)
• RHEL/CentOS 7 (el7)
• RHEL/CentOS/Alma/Rocky 8 (el8)

Note: the +6.g38b50cb version suffix is introduced by packaging automation added after the 3.9.1 release. There are no code/functionality changes vs the 3.9.1 source code.

SingularityCE 3.9.0

This is the first release of SingularityCE 3.9, the Community Edition of the Singularity container runtime hosted at https://github.com/sylabs/singularity.

Changed defaults / behaviours

• Building SingularityCE 3.9.0 requires go >=1.16. We now aim to support the two most recent stable versions of Go. This corresponds to the Go Release Maintenance Policy and Security Policy, ensuring critical bug fixes and security patches are available for all supported language versions.
• LABELs from Docker/OCI images are now inherited. This fixes a longstanding regression from Singularity 2.x. Note that you will now need to use --force in a build to override a label that already exists in the source Docker/OCI container.
• The source paths for %files lines in a definition file are no longer interpreted by a shell. This means that environment variable substitution is
  not performed. Previously, environment variables were substituted for source paths, but not destination paths, leading to unexpected copy behaviour. Globbing for source files will now follow the Go filepath.Match pattern syntax.
• Removed --nonet flag, which was intended to disable networking for in-VM execution, but has no effect.
• --nohttps flag has been deprecated in favour of --no-https. The old flag is still accepted, but will display a deprecation warning.
• Paths for cryptsetup, go, ldconfig, mksquashfs, nvidia-container-cli, unsquashfs are now found at build time by mconfig and written into singularity.conf. The path to these executables can be overridden by changing the value in singularity.conf.
• When calling ldconfig to find GPU libraries, singularity will not fall back to /sbin/ldconfig if the configured ldconfig errors. If installing in a Guix/Nix on environment on top of a standard host distribution you must set ldconfig path = /sbin/ldconfig to use the host distribution ldconfig to find GPU libraries.
• --nv will not call nvidia-container-cli to find host libraries, unless the new experimental GPU setup flow that employs nvidia-container-cli for all GPU related operations is enabled (see below).
• If a container is run with --nvcli and --contain, only GPU devices specified via the NVIDIA_VISIBLE_DEVICES environment variable will be exposed within the container. Use NVIDIA_VISIBLE_DEVICES=all to access all GPUs inside a container run with --nvccli.
• Example log-plugin rewritten as a CLI callback that can log all commands executed, instead of only container execution, and has access to command arguments.
• The bundled reference CNI plugins are updated to v1.0.1. The flannel plugin is no longer included, as it is maintained as a separate plugin at: https://github.com/flannel-io/cni-plugin. If you use the flannel CNI plugin you should install it from this repository.
• Instances are no longer created with an IPC namespace by default. An IPC namespace can be specified with the -i|--ipc flag.
• The behaviour of the allow container directives in singularity.conf has been modified, to support more intuitive limitations on the usage of SIF and non-SIF container images. If you use these directives, you may need to make changes to singularity.conf to preserve behaviour.
  • A new allow container sif directive permits or denies usage of unencrypted SIF images, irrespective of the filesystem(s) inside the SIF.
  • The allow container encrypted directive permits or denies usage of SIF images with an encrypted root filesystem.
  • The allow container squashfs/extfs directives in singularity.conf permit or deny usage of bare SquashFS and EXT image files only.
  • The effect of the allow container dir directive is unchanged.

New features / functionalities

• --writable-tmpfs can be used with singularity build to run the %test section of the build with a ephemeral tmpfs overlay, permitting tests that write to the container filesystem.
• The --compat flag for actions is a new short-hand to enable a number of options that increase OCI/Docker compatibility. Infers --containall, --no-init, --no-umask, --writable-tmpfs. Does not use user, uts, or network namespaces as these may not be supported on many installations.
• remote add --insecure may be used to configure endpoints that are only accessible via http.
• The experimental --nvccli flag will use nvidia-container-cli to setup the container for Nvidia GPU operation. SingularityCE will not bind GPU libraries itself. Environment variables that are used with Nvidia's docker-nvidia runtime to configure GPU visibility / driver capabilities & requirements are parsed by the --nvccli flag from the environment of the calling user. By default, the compute and utility GPU capabilities are configured. The use nvidia-container-cli option in singularity.conf can be set to yes to always use nvidia-container-cli when supported. Note that in a setuid install, nvidia-container-cli will be run as root with required ambient capabilities. --nvccli is not currently supported in the hybrid fakeroot (setuid install + --fakeroot) workflow. Please see documentation for more details.
• The --apply-cgroups flag can be used to apply cgroups resource and device restrictions on a system using the v2 unified cgroups hierarchy. The resource restrictions must still be specified in the v1 / OCI format, which will be translated into v2 cgroups resource restrictions, and eBPF device restrictions.
• A new --mount flag and SINGULARITY_MOUNT environment variable can be used to specify bind mounts in type=bind,source=<src>,destination=<dst>[,options...] format. This improves CLI compatibility with other runtimes, and allows binding paths containing : and , characters (using CSV style escaping).
• Perform concurrent multi-part downloads for library:// URIs. Uses 3 concurrent downloads by default, and is configurable in singularity.conf or via environment variables.

Bug fixes

• The oci commands will operate on systems that use the v2 unified cgroups hierarchy.
• Ensure invalid values passed to config global --set cannot lead to an empty configuration file being written.
• An invalid remote build source (bootstrap) will be identified before attempting to submit the build.
• --no-https now applies to connections made to library services specified in library://<hostname>/... URIs.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Please use the singularity-ce-3.9.0.tar.gz download below to obtain and install SingularityCE 3.9.0. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

SingularityCE 3.9.0 Release Candidate 3

This is the third release candidate for the upcoming SingularityCE 3.9.0. We'd be grateful for all testing, bug reports, and comments, as we look forward to a stable 3.9.0 release. Please carefully review the release notes below, and refer to the 'master branch (unreleased)' documentation at https://sylabs.io/docs/

This is a release candidate for SingularityCE 3.9.0

Changed defaults / behaviours

• The behaviour of the allow container directives in singularity.conf has been modified, to support more intuitive limitations on the usage of SIF and non-SIF container images. If you use these directives, you may need to make changes to singularity.conf to preserve behaviour.
  • A new allow container sif directive permits or denies usage of unencrypted SIF images, irrespective of the filesystem(s) inside the SIF.
  • The allow container encrypted directive permits or denies usage of SIF images with an encrypted root filesystem.
  • The allow container squashfs/extfs directives in singularity.conf permit or deny usage of bare SquashFS and EXT image files only.
  • The effect of the allow container dir directive is unchanged.

New features

• Perform concurrent multi-part downloads for library:// URIs. Uses 3 concurrent downloads by default, and is configurable in singularity.conf or via environment variables.

Bug fixes

• Ensure invalid values passed to config global --set cannot lead to an empty configuration file being written.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Please use the singularity-ce-3.9.0-rc.3.tar.gz download below to obtain and install SingularityCE 3.9.0-rc.3. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

SingularityCE 3.9.0 Release Candidate 2

This is the second release candidate for the upcoming SingularityCE 3.9.0. We'd be grateful for all testing, bug reports, and comments, as we look forward to a stable 3.9.0 release. Please carefully review the release notes below, and refer to the 'master branch (unreleased)' documentation at https://sylabs.io/docs/

Security related fixes

• Due to trusting a path to an executable that was incorrectly generated in code that could be manipulated by an unprivileged user, privilege escalation was possible when using the new --nvccli GPU configuration option. This vulnerability affected the 3.9.0-rc.1 release candidate only. Stable releases of SingularityCE are not impacted.

  All users who have installed 3.9.0-rc.1 should update to 3.9.0-rc.2

  Thanks to @cclerget for reporting this issue.

Changed defaults / behaviours

• The location of the cryptsetup, ldconfig and nvidia-container-cli binaries are always taken from singularity.conf. No $PATH search is performed.

Bug fixes

• Ensure a build with --nvccli runs using nvidia-container-cli and not the legacy gpu support.
• Advise on limitations and provide workaround for inability to run %test in --fakeroot --nvccli builds.

Additionally, this RC includes fixes introduced in SingularityCE 3.8.4

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Please use the singularity-ce-3.9.0-rc.2.tar.gz download below to obtain and install SingularityCE 3.9.0-rc.2. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

SingularityCE 3.8.4

This is a bugfix release of SingularityCE, the Community Edition of the Singularity container runtime hosted at https://github.com/sylabs/singularity. Documentation is available at https://sylabs.io/docs/.

Bug fixes

• Update oras-go dependency to address push failures to some registry configurations.
• Implement context cancellation when a signal is received in several CLI commands.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Please use the singularity-ce-3.8.4.tar.gz download below to obtain and install SingularityCE 3.8.4. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.

SingularityCE 3.9.0 Release Candidate 1

This is the first release candidate for the upcoming SingularityCE 3.9.0. We'd be grateful all testing, bug reports, and comments, as we look forward to a stable 3.9.0 release.

Various behavior changes and new features have been introduced. Please carefully review the release notes below, and refer to the 'master branch (unreleased)' documentation at https://sylabs.io/docs/

Changed defaults / behaviours

• Building SingularityCE 3.9.0 requires go >=1.16. We now aim to support the two most recent stable versions of Go. This corresponds to the Go Release Maintenance Policy and Security Policy, ensuring critical bug fixes and security patches are available for all supported language versions.
• LABELs from Docker/OCI images are now inherited. This fixes a longstanding regression from Singularity 2.x. Note that you will now need to use --force in a build to override a label that already exists in the source Docker/OCI container.
• The source paths for %files lines in a definition file are no longer interpreted by a shell. This means that environment variable substitution is not performed. Previously, environment variables were substituted for source paths, but not destination paths, leading to unexpected copy behaviour. Globbing for source files will now follow the Go filepath.Match pattern
  syntax.
• Removed --nonet flag, which was intended to disable networking for in-VM execution, but has no effect.
• --nohttps flag has been deprecated in favour of --no-https. The old flag is still accepted, but will display a deprecation warning.
• Paths for cryptsetup, go, ldconfig, mksquashfs, nvidia-container-cli, unsquashfs are now found at build time by mconfig and written into singularity.conf. The path to these executables can be overridden by changing the value in singularity.conf. If the path is not set in singularity.conf then the the executable will be found by searching $PATH.
• When calling ldconfig to find GPU libraries, singularity will not fall back to /sbin/ldconfig if the ldconfig on $PATH errors. If installing in a Guix/Nix on environment on top of a standard host distribution you must set ldconfig path = /sbin/ldconfig to use the host distribution ldconfig to find GPU libraries.
• --nv will not call nvidia-container-cli to find host libraries, unless the new experimental GPU setup flow that employs nvidia-container-cli for all GPU related operations is enabled (see below).
• If a container is run with --nvcli and --contain, only GPU devices specified via the NVIDIA_VISIBLE_DEVICES environment variable will be exposed within the container. Use NVIDIA_VISIBLE_DEVICES=all to access all GPUs inside a container run with --nvccli.
• Example log-plugin rewritten as a CLI callback that can log all commands executed, instead of only container execution, and has access to command arguments.
• An invalid remote build source (bootstrap) will be identified before attempting to submit the build.
• The bundled reference CNI plugins are updated to v1.0.1. The flannel plugin is no longer included, as it is maintained as a separate plugin at: https://github.com/flannel-io/cni-plugin. If you use the flannel CNI plugin you should install it from this repository.
• Instances are no longer created with an IPC namespace by default. An IPC namespace can be specified with the -i|--ipc flag.

New features / functionalities

• --writable-tmpfs can be used with singularity build to run the %test section of the build with a ephemeral tmpfs overlay, permitting tests that write to the container filesystem.
• --compat flag for actions is a new short-hand to enable a number of options that increase OCI/Docker compatibility. Infers --containall, --no-init, --no-umask, --writable-tmpfs. Does not use user, uts, or network namespaces as these may not be supported on many installations.
• --no-https now applies to connections made to library services specified in --library://<hostname>/... URIs.
• remote add --insecure may be used to configure endpoints that are only accessible via http.
• The experimental --nvccli flag will use nvidia-container-cli to setup the container for Nvidia GPU operation. SingularityCE will not bind GPU libraries itself. Environment variables that are used with Nvidia's docker-nvidia runtime to configure GPU visibility / driver capabilities & requirements are parsed by the --nvccli flag from the environment of the calling user. By
  default, the compute and utility GPU capabilities are configured. The use nvidia-container-cli option in singularity.conf can be set to yes to always use nvidia-container-cli when supported. Note that in a setuid
  install, nvidia-container-cli will be run as root with required ambient capabilities. --nvccli is not currently supported in the hybrid fakeroot (setuid install + --fakeroot) workflow. Please see documentation for more details.
• The --apply-cgroups flag can be used to apply cgroups resource and device restrictions on a system using the v2 unified cgroups hierarchy. The resource restrictions must still be specified in the v1 / OCI format, which will be translated into v2 cgroups resource restrictions, and eBPF device restrictions.
• A new --mount flag and SINGULARITY_MOUNT environment variable can be used to specify bind mounts in type=bind,source=<src>,destination=<dst>[,options...] format. This improves CLI compatibility with other runtimes, and allows binding paths containing : and , characters (using CSV style escaping).

Bug fixes

• The oci commands will operate on systems that use the v2 unified cgroups hierarchy.

Thanks / Reporting Bugs

Thanks to our contributors for code, feedback and, testing efforts!

As always, please report any bugs to: https://github.com/sylabs/singularity/issues/new

If you think that you've discovered a security vulnerability please report it to: security@sylabs.io

Have fun!

Downloads

Please use the singularity-ce-3.9.0-rc.1.tar.gz download below to obtain and install SingularityCE 3.9.0-rc.1. The GitHub auto-generated 'Source Code' downloads do not include required dependencies etc.