-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Add dev container configuration #12306
base: main
Are you sure you want to change the base?
Conversation
.devcontainer/Dockerfile
Outdated
@@ -0,0 +1,3 @@ | |||
# This Dockerfile is currently only used to enforce use of x86 image on ARM64 CPUs (Apple Silicon) | |||
# See aarch64 migration on https://conda-forge.org/status/, which is currently blocking us. | |||
FROM --platform=amd64 mcr.microsoft.com/devcontainers/base:debian-11 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
enforcing AMD64 platform here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why base:debian-11
instead of base:ubuntu-22.04
? My undestanding is that it can be very painful to get some packages (latest or not) in Debian.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ubuntu 22 and Debian 12 ship with OpenSSL 3, which dropped support for some older, insecure algorithms.
Our corporate firewall still injects some of these older certs in some places, so certain encrypted connections fail to be established. Sucks but that's how it is currently, not just for me.
All packages we need seem to be available in recent-enough versions. Is there anything particular you expect to be problematic? Remember we're not trying to provide an all-purpose dev environment here, only one specifically tailored for MNE development.
Many tools are available as dev container features too, so we're somewhat independent from the base OS anyway. And we can install binaries nova conda.
.devcontainer/devcontainer.json
Outdated
"packages": "mesa-utils,libegl1,^libxcb.*-dev,libx11-xcb-dev,libglu1-mesa-dev,libxrender-dev,libxi-dev,libxkbcommon-dev,libxkbcommon-x11-dev" | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could probably be trimmed down a little? not sure.
9ef71dd
to
edd8c9f
Compare
The password for VNC access is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this idea / initiative! I've left mostly questions and a few comments, so I can understand better how the config works.
.devcontainer/devcontainer.json
Outdated
"build": { "dockerfile": "Dockerfile" }, | ||
"containerEnv": { | ||
"PYTHONNOUSERSITE": "true", // Make Python ignore the user's site-packages folder if it exists | ||
"XDG_RUNTIME_DIR": "/home/vscode/.cache/xdgr" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I stumbled on this line, until I realized that "vscode" is being set as the username in the container. Will it still work if we change it to "mneuser" or similar (here and on line 131)? I.e., something that is more obviously an (arbitrary) username?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The base image I'm using sets up the vscode
user by default. There's been discussions on changing this to something more generic like devcontainer
, but this still remains to be implemented:
Changing it to a custom name like mneuser
would require changes to the Dockerfile and potentially some of the features we're using; hence I'd like to refrain from doing that โ there's no need to, you basically never deal with that username anyway outside of this very line here :)
I however changed the VNC password to mne
in 54b159b
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I may have found a workaround but I need to test it a bit first. It may make it harder for me to get the dev container to work behind our corporate firewall, where I need to inject our self-signed SSL certs via a custom feature first. I want to make sure this doesn't become even more difficult or impossible if I change the default user name here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok but don't spend too much effort on it, it's a minor maintainer-facing improvement right (user will not notice)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's basically irrelevant to users of the container, yes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Username is now mne-user
.devcontainer/devcontainer.json
Outdated
// Zsh plugins | ||
"ghcr.io/devcontainers-contrib/features/zsh-plugins:0": { | ||
"plugins": "zsh-autosuggestions zsh-syntax-highlighting history-substring-search", | ||
"omzPlugins": "https://github.com/zsh-users/zsh-autosuggestions https://github.com/zsh-users/zsh-syntax-highlighting" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why are these needed? Will the user ever access a terminal in the container (and if so, won't it be a BASH terminal by default?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, of course :) It's a complete development environment, and if you open a terminal, it will open a "remote" shell inside the Docker container.
The default shell is zsh with oh-my-zsh.
I'm trying to provide a nice and comfy shell experience through these plugins.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm trying to provide a nice and comfy shell experience through these plugins.
Another word for "comfy" is "familiar". I don't necessarily object to including zsh
alongside bash
but I'm dubious about making zsh
the default here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MacOS users have zsh
as the default shell and zsh
is very close to bash
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, I know that since 2019 macOS uses zsh as the default shell. And I've heard that they're similar. What I'm questioning is why tailor the experience in a linux container to feel familiar to macOS users. Sticking with BASH seems like the natural default choice, and deviation from that feels like it needs to be justified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both shells are first-class citizens in the MS dev containers. By default, both are shipped, hence I'm shipping both too. The plugins listed here make the zsh experience better and superior to the Bash experience. I have not yet decided which shell shall be the default one. However, I'm surprised this seems to be so controversial. VS Code clearly displays which shell is being used, and when spawning a new terminal, one can select the shell from a dropdown menu. Besides, I don't see how the choice of shell matters as long as one doesn't do shell programming; and this is an MNE dev container, no generic Linux Shell Programming dev container. Like I said, I haven't decided yet which default I think would be best; but It's quite obvious to me though that the zsh prompt looks better and the shell provides more help during interactive use (history substring search being one of those invaluable features; git support being another).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not yet decided which shell shall be the default one.
This is news to me; you originally set zsh
as default (which I noticed at the time), then removed that setting in 7d5c6a2 after my earlier comments. It seems a bit disingenuous to now say "why is this so controversial, they're both included and I haven't chosen a default".
That said, I'm happy if both are included and it's easy (and documented) for users to switch the shell they get in their container. Ideally this would be a one-time config (not something that needs doing every time the container is launched) if that's possible
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is news to me; you originally set
zsh
as default (which I noticed at the time), then removed that setting in 7d5c6a2 after my earlier comments
Ah, I see!! No, this was actually a misunderstanding. The default login shell always was Bash, and the setting you're referring to controls which shell should be spawned by default when creating a new terminal without explicitly selecting a shell. This was in fact an accidental setting that I copied over from another dev container config I'm working with. But this setting never changed the (login) default shell to zsh, nor dropped bash or something :) I had however forgotten to properly set up mamba
and conda
for bash โย something I realized and fixed after your comment / question.
That said, I'm happy if both are included and it's easy (and documented) for users to switch the shell they get in their container. Ideally this would be a one-time config (not something that needs doing every time the container is launched) if that's possible
Yep, I'm open for this! for now I'm really still trying to make things work and experimenting a bit! Let's save the heated debates for later! ๐๐ค
Oh and, happy holidays to you all!!!
// Configure properties specific to VS Code. | ||
"vscode": { | ||
"settings": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seems like fertile ground for bikeshedding. Is there a way for the container to pick up the user's local settings? If not, then can you indicate which of these settings are truly necessary for the container to work, which are not necessary but clearly a good idea (e.g., don't restore port forwards?), and which are your preferences (perhaps "format on type" fits here)?
.gitignore
Outdated
@@ -97,6 +97,7 @@ cover | |||
.venv/ | |||
venv/ | |||
*.json | |||
!/.devcontainer/devcontainer.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
!/.devcontainer/devcontainer.json | |
!.devcontainer/devcontainer.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually wanted to be explicit, hence the "full" path specification here
Thanks for you feedback, @drammock |
would be good to get this "into" 1.7, but i'm not sure if i'll be able to finish it in time i know it's devs-only but it would be nice to have a reference to this in the 1.7 stable docs |
In this case let's not mark it for 1.7 -- let's only mark the things that are blockers for release (which should happen this week ideally). If it gets merged in time though that's great (nothing stopping it if you can get it done)! |
ok I'm fine with that! |
๐ฉโ๐ป The experience I'm implementing here
Screen.Recording.2023-12-18.at.15.29.08.mov
๐ Background
In the past year, I moved almost all my development work into dev containers.
Until recently, one of the few development workflows I had not yet transitioned to use dev containers was MNE-Python. Reason was that the interactive figures don't work well in a notebook or web browser, and I didn't want to run an X server on my host just to display the figures.
๐ก Solution
I created a dev container configuration that deploys a remote resktop feature. It allows for a remote connection via a VNC client, or via the web browser. Browser-based access even works inside VS Code with the built-in Simple Browser.
The dev container runs on Debian 11, installs MNE-python via
conda-forge
, followed by apip
-based editable install of the version in the checked-out repository.๐ฏ Scope & target audience
๐ฎ Future plans
sample
andtesting
datasets.๐โโ๏ธ Using the dev container
You need
Open the repository in VS Code and click on Reopen in Container in the bottom right.
The Debian docker image and dev container features including MNE will be downloaded and installed. Lastly, an editable install of MNE will be carried out.
The VNC server will be exposed on
localhost:5901
(you can connect to it with any VNC client, e.g.Screen Sharing.app
on macOS), and the web server can be reached onhttp://localhost:6080
. The password isvscode
.You can now start developing and using MNE-Python. Interactive figures will appear on the screen exposed via the VNC server.
All development dependencies are installed (incl. pre-commit hooks, pytest etc.), and the editor is set to auto-format code via Ruff on saving.
The host SSH agent's socket is transparently forwarded into the container, meaning you should be able to use SSH immediately without requiring any further configuration.
The host's git configuration is copied to the container as well, so your name and email address are set correctly, too.
โ Limitations
I currently enforce running an AMD64 Debian image on all platforms, including Apple Silicon. The native ARM64 wouldn't work as MNE-Python cannot be installed from
conda-forge
on this platform, as the migration ofaarch64
dependencies is still ongoing. To avoid this, all deps could be potentially installed from PyPI viapip
, but I haven't tried this yet.โThoughts?
What do you all think?
cc @cbrnr @britta-wstnr @agramfort @sappelhoff