Python Devcontainer fails to start

[imrehg]

Topic

devcontainer vscode

Relevant information

When I've tried to use the included devcontainer setup for Python, I couldn't get it started (container build fails).

From the log it's not really clear what exactly is happening, but this seems

[...snip...] 
 => => # error: invalid command 'bdist_wheel'                                  
 => => # ERROR: Feature "Python" (ghcr.io/devcontainers/features/python) failed
 => => #  to install! Look at the documentation at https://github.com/devcontai
 => => # ners/features/tree/main/src/python for help troubleshooting this error
 => => # .   
[...snip...] 
 [2024-05-17T04:50:58.365Z] Stop (75901 ms): Run: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Users/gergely/.vscode/extensions/ms-vscode-remote.remote-containers-0.362.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/gergely/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-31f3eced-24f5-4536-8be5-98432a9ad96a1715921381279 --workspace-folder /Users/gergely/faculty/airbyte --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/gergely/faculty/airbyte --id-label devcontainer.config_file=/Users/gergely/faculty/airbyte/.devcontainer/python-connectors-generic/devcontainer.json --log-level debug --log-format json --config /Users/gergely/faculty/airbyte/.devcontainer/python-connectors-generic/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-05-17T04:50:58.365Z] Exit code 1
[2024-05-17T04:50:58.367Z] Command failed: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Users/gergely/.vscode/extensions/ms-vscode-remote.remote-containers-0.362.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /Users/gergely/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-31f3eced-24f5-4536-8be5-98432a9ad96a1715921381279 --workspace-folder /Users/gergely/faculty/airbyte --workspace-mount-consistency cached --id-label devcontainer.local_folder=/Users/gergely/faculty/airbyte --id-label devcontainer.config_file=/Users/gergely/faculty/airbyte/.devcontainer/python-connectors-generic/devcontainer.json --log-level debug --log-format json --config /Users/gergely/faculty/airbyte/.devcontainer/python-connectors-generic/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-05-17T04:50:58.367Z] Exit code 1

Following up from a comment, it points at some bugs / incompatible changes in the devcontainer provided setup.

Possible fixes

Both of these I've tried and allows the dev container to start, so far only smoke-tested them.

Pinning the Python feature to an earlier working verison

airbyte/.devcontainer/python-connectors-generic/devcontainer.json

Line 8 in 6ea7957

"ghcr.io/devcontainers/features/python:1": {

to be replaced by the following to pin it to a working (tested) version:

    "ghcr.io/devcontainers/features/python:1.3": {

(the current latest version for that feature is 1.4.2, which appears to carry this bug.)

This seems to be the least amount of change, but also needs to be revisited later (to remove the pin when the bug is fixed).

Bumping the Python dev container to a newer version

airbyte/.devcontainer/python-connectors-generic/devcontainer.json

Line 5 in 6ea7957

"image": "mcr.microsoft.com/devcontainers/python:0-3.10",

replaced by:

  "image": "mcr.microsoft.com/devcontainers/python:1-3.10",

So far I couldn't find any release notes why the major version has been incremented (list of history), but this seems to work at first sight, without needing to pin the "feature" above to a non-latest version.

This seems to be the more forward-looking solution, but might need some initial validation due to the image major version bump.

I'm happy to submit patch & test either of these, or see the team handle it by their way, of course. :)

[aaronsteers]

@imrehg - Thanks for posting this. We are experimenting with these devcontainers as a means of streamlining community contributions as well as maintainer's code review processes. Your proposals to bump the version sound good to me. If you are able to contribute a PR, please tag me as reviewer and I'll happy review and approve.

We could also consider reverting to the "universal" image that Github/Microsoft are maintaining - while it is a larger image, it comes pre-loaded with Python and Node (to host VS Code and its extensions) and the universal image appears to be cached on runners and has a faster build times. (Not based on controlled experiments, just based on anecdotal observations.) On the negative side, I've had sporadic problems git committing (PGP error) on the universal image, although that may have been a one-time fluke.

Long-story short, we welcome your suggestions and I appreciate your posting this issue. Lmk how I can help! 🙏

[imrehg]

@aaronsteers I've submitted a PR with the proposed second option, to minimise the changes and focus on making the environment better fit for purpose.

Now that I can run the dev container, I can see that that the actual developer flow is indeed quite experimental in there, partly because it's new to me (so I'm not sure if everything has been set up correctly), and partly because it seems like the container does make the various config more mixed up (mapping config from the dev machine into the cotnainer).

As some illustrative examples, I use ssh key signing, and most config is loaded fine except the allowed signing keys file, since it's referring to the location outside of the container. Also had some issues getting the pre-commit hook work (I didn't). These bits are something to explore in a separate PR, probably. And that motivated me to keep this, rather than switching to the universal image, as I wouldn't be sure which issues that I encounter is due to the image change versus all the other setup. I can see that move would simplify the setup though, so likely it's a fitting suggestion for later.