[CUDA-enabled] Data Science Dev Containers

[GPU accelerated] Multi-arch (linux/amd64, linux/arm64/v8) Data Science Dev Containers:

• [CUDA] Julia base, pubtools
• Mojo base, scipy
• [CUDA] Python base, scipy
• [CUDA] R base, tidyverse, verse, geospatial, qgisprocess

Dev Containers considered stable for

• Julia versions ≥ 1.7.3
• Mojo versions ≥ 24.3.0
• Python versions ≥ 3.10.5
• R versions ≥ 4.2.0

Parent images

• [CUDA-enabled] Julia docker stack
• Mojo docker stack
• [CUDA-enabled] Python docker stack
• [CUDA-enabled] R docker stack

Extended to match the [CUDA-enabled] JupyterLab docker stacks, except that

• GPU accelerated Dev Containers are based on the NVIDIA CUDA runtime flavoured image.
  • The JupyterLab docker stacks are based on the NVIDIA CUDA devel flavoured image.
• Dev Containers' Oh My Zsh uses the devcontainers theme + default font.
  • The JupyterLab docker stacks' Oh My Zsh uses Powerlevel10k theme + MesloLGS NF font.

Features

• JupyterLab: A web-based interactive development environment for Jupyter notebooks, code, and data.
• Git: A distributed version-control system for tracking changes in source code.
• Git LFS: A Git extension for versioning large files.
• GRASS GIS: A free and open source Geographic Information System (GIS).
  ℹ️ R qgisprocess image
• Orfeo Toolbox: An open-source project for state-of-the-art remote sensing.
  ℹ️ R qgisprocess image (amd64 only)
• Julia¹: A high-level, high-performance dynamic language for technical computing.
• Mojo¹: A programming language for AI developers.
• Pandoc: A universal markup converter.
• Python: An interpreted, object-oriented, high-level programming language with dynamic semantics.
• QGIS: A free, open source, cross platform (lin/win/mac) geographical information system (GIS).
  ℹ️ R qgisprocess image
• Quarto: A scientific and technical publishing system built on Pandoc.
  ℹ️ Julia pubtools, Mojo/Python scipy, R verse+ images
• R¹: A language and environment for statistical computing and graphics.
• SAGA GIS: A Geographic Information System (GIS) software with immense capabilities for geodata processing and analysis.
  ℹ️ R qgisprocess image
• TinyTeX: A lightweight, cross-platform, portable, and easy-to-maintain LaTeX distribution based on TeX Live.
  ℹ️ Julia pubtools, Mojo/Python scipy, R verse+ images
• Zsh: A shell designed for interactive use, although it is also a powerful scripting language.

👉 See the Version Matrices for detailed information:

• Julia: Version Matrix, CUDA Version Matrix
• Mojo: Version Matrix
• Python: Version Matrix, CUDA Version Matrix
• R: Version Matrix, CUDA Version Matrix

Pre-installed extensions

• .gitignore Generator
• Black Formatter
• Docker
• EditorConfig for VS Code
• ESLint
• Git Graph
• GitHub Pull Requests and Issues
• GitLab Workflow
• GitLens — Git supercharged
  ℹ️ Pinned to version 11.7.0 due to unsolicited AI content
• Excel Viewer
• hadolint
• Julia¹
• Jupyter
• LaTeX Workshop
  ℹ️ Julia pubtools, Mojo/Python scipy, R verse+ images
• markdownlint
• Mojo¹
• Path Intellisense
• Prettier - Code formatter
• Project Manager
• Python
• Quarto
  ℹ️ Julia pubtools, Mojo/Python scipy, R verse+ images
• R¹
• Resource Monitor
• ShellCheck
• YAML

Table of Contents

• Prerequisites
• Install
• Usage
• Similar project
• Contributing
• License

Prerequisites

Dev Containers require either Docker or Podman² to be installed. CUDA-enabled versions require the following in addition:

• NVIDIA GPU
• NVIDIA Linux driver
• NVIDIA Container Toolkit

ℹ️ The host running the GPU accelerated Dev Containers only requires the NVIDIA driver, the CUDA toolkit does not have to be installed.

Install

Codespaces require no installation, but do not currently offer machines with NVIDIA GPUs.

Docker

To install Docker, follow the instructions for your platform:

• Install Docker Engine | Docker Documentation > Supported platforms
• Post-installation steps for Linux

Podman

To install Podman, follow the instructions for your platform:

• Podman Installation | Podman

CUDA

To install the NVIDIA Container Toolkit, follow the instructions for your platform:

• Installation Guide — NVIDIA Cloud Native Technologies documentation

Usage

The default Dev Container is meant to work on this repository.

Every other configuration is a custom Data Science Dev Container that behaves in a unique way:

1. Default mount³:
   • source: empty directory
   • target: /home/vscode
   • type: volume
2. Codespace only mount:
   • source: root of this repository
   • target: /workspaces
   • type: misc
3. Default path: /home/vscode
4. Default user⁴: vscode
   • uid: 1000 (auto-assigned)
   • gid: 1000 (auto-assigned)
5. Lifecycle scripts:
   • onCreateCommand: home directory setup
   • postStartCommands
     • docker: Silently remove all unused images and all build cache (Codespace only)
     • julia: Copy user-specific startup files
     • r: Copy QGIS stuff from skeleton directory; Create R user library
   • postAttachCommand: Codespace only: Check for Dev Container updates

To disable the postStartCommand or postAttachCommand, comment out line 8 in ~/.local/bin/dockerSystemPrune.sh or ~/.local/bin/checkForUpdates.sh.

Codespace

1. Click the <> Code button, then click the Codespaces tab.
   A message is displayed at the bottom of the dialog telling you who will pay for the codespace.
2. Create your codespace after configuring advanced options:
   • Configure advanced options
     To configure advanced options for your codespace, such as a different machine type or a particular devcontainer.json file:
     • At the top right of the Codespaces tab, select ... and click New with options....
     • On the options page for your codespace, choose your preferred options from the dropdown menus.
     • Click Create codespace.

– Creating a codespace for a repository - GitHub Docs

To open your codespace in JupyterLab:

1. Execute

   jupyter-lab \
     --ServerApp.allow_origin='*' \
     --ServerApp.cookie_options="{'Same Site': 'None', 'Secure': True}" \
     --ServerApp.tornado_settings="{'headers':{'Content-Security-Policy':\"frame-ancestors 'self' https://*.github.dev\", 'Access-Control-Allow-Headers': 'accept, content-type, authorization, x-xsrftoken, x-github-token'}}" \
     --notebook-dir=/home/vscode \
     --no-browser
   

2. Ctrl+click on one of the URLs shown in the Terminal.

ℹ️ Opening your codespace in JupyterLab according to the GitHub Docs sets the default path to /workspaces/<repository-name> that you can not escape.

Local/'Remote SSH'

Use the Dev Containers: Reopen in Container command from the Command Palette (F1, ⇧⌘P (Windows, Linux Ctrl+Shift+P))

To start JupyterLab:

1. Execute

   jupyter-lab
   

2. Ctrl+click on one of the URLs shown in the Terminal.

Persistence

Data in the following locations is persisted:

1. The user's home directory (/home/vscode)⁵
2. The Dev Container's workspace (/workspaces)

This is accomplished either via a volume or bind mount (or loop device on Codespaces) and is preconfigured.

Codespaces: A 'Full Rebuild Container' resets the home directory! ℹ️ This is never necessary unless you want exactly that.

Similar project

• rocker-org/devcontainer-images

What makes this project different:

1. Multi-arch: linux/amd64, linux/arm64/v8
   ℹ️ Runs on Apple M series using Docker Desktop.
2. Base image: Debian instead of Ubuntu
   ℹ️ CUDA-enabled images are Ubuntu-based.
3. IDE: JupyterLab next to VS Code
4. Just Python – no Conda / Mamba

CUDA-enabled images:

1. Derived from nvidia/cuda:12.4.1-runtime-ubuntu22.04
2. TensortRT and TensorRT plugin libraries

See Notes and CUDA Notes for tweaks, settings, etc.

Contributing

PRs accepted.

This project follows the Contributor Covenant Code of Conduct.

License

Copyright © 2023 b-data GmbH

Distributed under the terms of the MIT License.

Footnotes

1. Depending on which Dev Container configuration is selected. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

2. See issue #1 about limitations in Podman. ↩

3. See issue #2 about changing the mount type. ↩

4. See issue #3 about running as root. ↩

5. Alternatively for the root user (/root). Use with Docker/Podman in rootless mode. ↩