Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved support for running as non-root #503

Merged
merged 8 commits into from Mar 21, 2024
Merged

Improved support for running as non-root #503

merged 8 commits into from Mar 21, 2024

Conversation

ikreymer
Copy link
Member

@ikreymer ikreymer commented Mar 21, 2024

This PR provides improved support for running crawler as non-root, matching the user to the uid/gid of the crawl volume.

This fixes #502 initial regression from 0.12.4, where chmod u+x was used instead of chmod a+x on the node binary files.

However, that was not enough to fully support equivalent signal handling / graceful shutdown as when running with the same user. To make the running as different user path work the same way:

  • need to switch to gosu instead of su (added in Brave 1.64.109 image)
  • run all child processes as detached (redis-server, socat, wacz, etc..) to avoid them automatically being killed via SIGINT/SIGTERM
  • running detached is controlled via DETACHED_CHILD_PROC=1 env variable, set to 1 by default in the Dockerfile (to allow for overrides just in case)

A test has been added which runs one of the tests with a non-root test-crawls directory to test the different user path. The test (saved-state.test.js) includes sending interrupt signals and graceful shutdown and allows testing of those features for a non-root gosu execution.

Also bumping crawler version to 1.0.1

@ikreymer ikreymer requested a review from tw4l March 21, 2024 06:32
Copy link
Contributor

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Appreciate the extra test case, and graceful stopping is working much better now.

@ikreymer ikreymer merged commit 1fe810b into main Mar 21, 2024
4 checks passed
@ikreymer ikreymer deleted the dockerfile-fix branch March 21, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

/usr/bin/crawl Permission denied
2 participants