Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic VS Code dev container config #4517

Draft
wants to merge 3 commits into
base: maintenance
Choose a base branch
from

Conversation

Mearman
Copy link
Member

@Mearman Mearman commented May 22, 2022

  • Your changes are not possible to do through a plugin and relevant
    to a large audience (ideally all users of OctoPrint)
  • If your changes are large or otherwise disruptive: You have
    made sure your changes don't interfere with current development by
    talking it through with the maintainers, e.g. through a
    Brainstorming ticket
  • Your PR targets OctoPrint's devel branch if it's a completely
    new feature, or maintenance if it's a bug fix or improvement of
    existing functionality for the current stable version (no PRs
    against master or anything else please)
  • Your PR was opened from a custom branch on your repository
    (no PRs from your version of master, maintenance or devel please),
    e.g. dev/my_new_feature or fix/my_bugfix
  • Your PR only contains relevant changes: no unrelated files,
    no dead code, ideally only one commit - rebase and squash your PR
    if necessary!
  • Your changes follow the existing coding style
  • If your changes include style sheets: You have modified the
    .less source files, not the .css files (those are generated with
    lessc)
  • You have tested your changes (please state how!) - ideally you
    have added unit tests
  • You have run the existing unit tests against your changes and
    nothing broke
  • You have added yourself to the AUTHORS.md file :)

What does this PR do and why is it necessary?

Adds initial VS Code config for dev container environment and launch/task config.
Allows contributors to quickly spin up a clean environment for development on both OctoPrint and plugins

How was it tested? How can it be tested by the reviewer?

Any background context you want to provide?

What are the relevant tickets if any?

Screenshots (if appropriate)

Further notes

@github-actions github-actions bot added targets devel The PR targets the devel branch approved Issue has been approved by the bot or manually for further processing labels May 22, 2022
@Mearman Mearman marked this pull request as ready for review May 22, 2022 15:12
@Mearman Mearman force-pushed the dev_container branch 2 times, most recently from 9d599f8 to aa41215 Compare May 22, 2022 15:26
@cp2004
Copy link
Member

cp2004 commented May 22, 2022

Docker on Windows has just really annoyed me, but...

The pre-commit install seemed to fail:

An unexpected error has occurred: PermissionError: [Errno 1] Operation not permitted: '.git/hooks/pre-commit'
Check the log at /home/vscode/.cache/pre-commit/pre-commit.log
[207750 ms] postCreateCommand failed with exit code 3. Skipping any further user-provided commands.

And the contents of /home/vscode/.cache/pre-commit/pre-commit.log:

version information

pre-commit version: 2.19.0
git --version: git version 2.30.2
sys.version:
    3.10.4 (main, Apr  7 2022, 03:15:56) [GCC 10.2.1 20210110]
sys.executable: /usr/local/bin/python
os.name: posix
sys.platform: linux

error information

An unexpected error has occurred: PermissionError: [Errno 1] Operation not permitted: '.git/hooks/pre-commit'
Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/main.py", line 371, in main
    return install(
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/commands/install_uninstall.py", line 132, in install
    _install_hook_script(
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/commands/install_uninstall.py", line 110, in _install_hook_script
    make_executable(hook_path)
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/util.py", line 78, in make_executable
    os.chmod(filename, new_mode)
PermissionError: [Errno 1] Operation not permitted: '.git/hooks/pre-commit'

OctoPrint works, it is slow but I think it's being limited in resources (and the random background CPU of 10% by Docker Desktop 😠 doesn't help).

@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

interesting @cp2004 I'll spin up my Windows VM and test. could I get your Windows, Docker and maybe Git versions, please?
I assume you're not running it under WSL or anything?

@foosel do you get the same on Windows?

@foosel
Copy link
Member

foosel commented May 23, 2022

Haven't yet had a chance to check, but one thing - I think this should rather go into maintenance. It's not a new giant feature in OctoPrint but rather a QOL improvement of OctoPrint and OctoPrint-adjacent development through tooling, so it should rather target maintenance (and possibly should also see backporting to master).

@Mearman Mearman changed the base branch from devel to maintenance May 23, 2022 08:13
@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

No worries. Retargeted.
Would you prefer my branch to be rebased on maintenance as well @foosel?

@foosel
Copy link
Member

foosel commented May 23, 2022

Yes, otherwise any changes that already made it into devel will be merged to maintenance upon merge (at the moment that would be 157 commits instead of just your four-ish).

Also can confirm, after just setting up Docker again on this system (new PC, still finding missing bits and pieces, virtualization was even still disabled and so I just got best friends with the bios again XD) I run into the same issue that @cp2004 encountered. Permission error when pre-commit is attempting to install the pre-commit hook:


version information

pre-commit version: 2.19.0
git --version: git version 2.30.2
sys.version:
    3.10.4 (main, Apr  7 2022, 03:15:56) [GCC 10.2.1 20210110]
sys.executable: /usr/local/bin/python
os.name: posix
sys.platform: linux

error information

An unexpected error has occurred: PermissionError: [Errno 1] Operation not permitted: '.git/hooks/pre-commit'
Traceback (most recent call last):
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/main.py", line 371, in main
    return install(
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/commands/install_uninstall.py", line 132, in install
    _install_hook_script(
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/commands/install_uninstall.py", line 110, in _install_hook_script
    make_executable(hook_path)
  File "/home/vscode/.local/lib/python3.10/site-packages/pre_commit/util.py", line 78, in make_executable
    os.chmod(filename, new_mode)
PermissionError: [Errno 1] Operation not permitted: '.git/hooks/pre-commit'

I could imagine this is due to this checkout already being set-up in the local environment? So I did a quick check-out of a fresh copy and tried it again, and yep, that's it. So that needs some kind of handling.

Also, I think it would make more sense to not start OctoPrint right away but rather ship a ready made launch config so that debugging is easily possible?

TODOs/To-Figure-Outs I see here:

  • find way to handle pre-commit install issue - best bet is probably to fail gracefully & log a warning or something...
  • debugging launch config instead of auto-start, incl. auto update of dependencies
  • workspace needs to be configured to run linters and formatters (black, isort, flake8) on save
  • launch config for pre-commit for manual run?
  • can plugin sources be mounted inside the container easily for development of third party plugins or do we need a different kind of container for that?
  • is there a way to tell vscode to install pylance automatically for this container instead of asking during setup?

@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

fairly certain it's not constrained to Windows. It's that the hooks were installed by your user so the container doesn't have the permissions to update them.

To confirm, either @foosel or @cp2004, could you test by removing the hooks and then launching the dev container, please?

@foosel
Copy link
Member

foosel commented May 23, 2022

To confirm, either @foosel or @cp2004, could you test by removing the hooks and then launching the dev container, please?

Already did in a way, by doing a fresh checkout into a different folder and then it worked.

@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

Already did in a way, by doing a fresh checkout into a different folder and then it worked.

Excellent, I borked my Windows system so just restoring it and well try and reproduce, but that should be a fairly easy case to fix

@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

TODOs/To-Figure-Outs I see here:

  • find way to handle pre-commit install issue - best bet is probably to fail gracefully & log a warning or something...
  • debugging launch config instead of auto-start, incl. auto update of dependencies
  • workspace needs to be configured to run linters and formatters (black, isort, flake8) on save
  • launch config for pre-commit for manual run?
  • can plugin sources be mounted inside the container easily for development of third party plugins or do we need a different kind of container for that?
  • is there a way to tell vscode to install pylance automatically for this container instead of asking during setup?

Hahah, I've actually already got the majority of this in another branch. I separated it though as I didn't know how prescriptive you wanted the base container config to be. I'll look at bringing them in once I've found a a nice solution to the pre-commit install issue

@Mearman
Copy link
Member Author

Mearman commented May 23, 2022

@cp2004 or @foosel I've not been able to reproduce the pre-commit issues. In an environment where you're still seeing the issue, could one of you test switching the postCreateCommand for initializeCommand then onCreateCommand, please?

@cp2004
Copy link
Member

cp2004 commented May 23, 2022

@cp2004 or @ foosel I've not been able to reproduce the pre-commit issues. In an environment where you're still seeing the issue, could one of you test switching the postCreateCommand for initializeCommand then onCreateCommand, please?

Sorry, I won't have a chance to test probably until the weekend again. If I do, I'll let you know but it's not looking likely I'll get the chance to work on any OctoPrint related stuff properly for this week.

"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
"python.formatting.provider": "black",
"python.formatting.blackArgs": ["--config", "black.toml"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the merger of #4511, this is no longer needed.

Suggested change
"python.formatting.blackArgs": ["--config", "black.toml"],

Comment on lines +24 to +32
"python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
"python.formatting.blackPath": "/usr/local/py-utils/bin/black",
"python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
"python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
"python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
"python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
"python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
"python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
"python.linting.pylintPath": "/usr/local/py-utils/bin/pylint",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The downside of relying on the ones that came with the container is that there is no control over which versions it uses. Especially with black, mypy & pylint; a single version difference may change behavior quite a bit.

I would applaud/recommend pin/controlling QA tools on a project level; and installing and using them locally.

@foosel foosel marked this pull request as draft May 24, 2022 15:01
@foosel
Copy link
Member

foosel commented May 24, 2022

Took the liberty to convert this to draft/WIP since you are still adding to it.

@cp2004
Copy link
Member

cp2004 commented May 27, 2022

Have some spare time today to do some experimenting with this...

It seems that the permission errors are because the git checkout (& associated files) are done as root on the container, so the vscode user can't run pre-commit install:

/workspaces/OctoPrint (pr/4517 ✗) $ ls -l
total 185
-rwxrwxrwx 1 root root  8501 May 26 17:46 AUTHORS.md
-rwxrwxrwx 1 root root   160 Jan 27 18:04 CHANGELOG.md
-rwxrwxrwx 1 root root   103 Jan 27 18:04 CODE_OF_CONDUCT.md
-rwxrwxrwx 1 root root 20031 May 26 11:45 CONTRIBUTING.md
-rwxrwxrwx 1 root root 35181 Jan 27 18:04 LICENSE.txt
-rwxrwxrwx 1 root root   390 May 26 11:45 MANIFEST.in
-rwxrwxrwx 1 root root  8758 May 26 11:45 README.md
-rwxrwxrwx 1 root root   305 Jan 27 18:04 SECURITY.md
-rwxrwxrwx 1 root root  1706 May 26 11:45 SUPPORTERS.md
-rwxrwxrwx 1 root root  5352 May 26 11:45 THIRDPARTYLICENSES.md
drwxrwxrwx 1 root root   512 May 26 17:54 __pycache__
-rwxrwxrwx 1 root root   498 May 26 11:45 babel.cfg
-rwxrwxrwx 1 root root   307 Jan 27 18:04 black.toml
drwxrwxrwx 1 root root   512 Mar 15 00:23 build
drwxrwxrwx 1 root root   512 Feb 27 21:01 dist
drwxrwxrwx 1 root root   512 May 26 17:46 docs
drwxrwxrwx 1 root root   512 May 26 11:29 htmlcov
-rwxrwxrwx 1 root root   147 Jan 27 18:04 pytest.ini
-rwxrwxrwx 1 root root   329 Jan 27 18:04 requirements.txt
drwxrwxrwx 1 root root   512 May 26 11:45 scripts
-rwxrwxrwx 1 root root   717 Jan 27 18:04 setup.cfg
-rwxrwxrwx 1 root root  9108 May 26 17:46 setup.py
drwxrwxrwx 1 root root   512 Jan 27 18:08 src
drwxrwxrwx 1 root root   512 May 26 11:45 tests
drwxrwxrwx 1 root root   512 May 26 11:45 translations
-rwxrwxrwx 1 root root 81111 May 26 17:46 versioneer.py

In the devcontainer.json file it has "remoteUser": "vscode" - so when it runs the commands, it is running as that user.

I got rid of this line, so it would log in as and run everything as root, and it worked (with adding --iknowwhatimdoing to OctoPrint's serve command (postAttachCommand)).

@foosel foosel added the wrong target The PR targets the wrong branch (e.g. `devel` instead of `maintenance`) label May 30, 2022
@cp2004 cp2004 removed targets devel The PR targets the devel branch wrong target The PR targets the wrong branch (e.g. `devel` instead of `maintenance`) labels Jun 13, 2022
@cp2004 cp2004 added the targets maintenance The PR targets the maintenance branch label Jun 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Issue has been approved by the bot or manually for further processing targets maintenance The PR targets the maintenance branch
Projects
Status: Blocked
Development

Successfully merging this pull request may close these issues.

None yet

4 participants