Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow jupyt command and env customization #536

Open
chegoryu opened this issue Apr 17, 2024 · 0 comments
Open

Allow jupyt command and env customization #536

chegoryu opened this issue Apr 17, 2024 · 0 comments

Comments

@chegoryu
Copy link
Contributor

chegoryu commented Apr 17, 2024

Right now command and env are hardcoded:

func (c *Controller) buildCommand(speclet *Speclet) (command string, env map[string]string) {
// TODO(max): take port from YT_PORT_0.
// TODO(max): come up with a solution how to pass secrets (token or password) without exposing them in the
// strawberry attributes.
cmd := fmt.Sprintf(
"bash -x start.sh /opt/conda/bin/jupyter lab --ip '*' --port %v --LabApp.token='' --allow-root >&2", JupytPort)
return cmd, map[string]string{
"NB_GID": "0",
"NB_UID": "0",
"NB_USER": "root",
}
}

Also there is no way to pass environment/secure_vault or add custom patch to spec:

spec = map[string]any{
"tasks": map[string]any{
"jupyter": map[string]any{
"command": command,
"job_count": 1,
"docker_image": speclet.JupyterDockerImage,
"memory_limit": speclet.MemoryOrDefault(),
"cpu_limit": speclet.CPUOrDefault(),
"port_count": 1,
"max_stderr_size": 1024 * 1024 * 1024,
"user_job_memory_digest_lower_bound": 1.0,
"environment": env,
},
},
"max_failed_job_count": 10 * 1000,
"max_stderr_count": 150,
"title": "JUPYT notebook *" + alias,
}
annotations = map[string]any{
"is_notebook": true,
"expose": true,
}

I have ugly workaround with custom docker image:

RUN mv /usr/local/bin/start.sh /usr/local/bin/real_start.sh
COPY my_start.sh /usr/local/bin/start.sh

Where I replace entrypoint command like this:

NB_USER=jovyan NB_UID=1000 NB_GID=100 PATH='/opt/conda/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/spark/bin' real_start.sh run-one-constantly jupyter lab --ip '*' --port 27042 '--NotebookApp.password=argon2:$argon2id$...' --ResourceUseDisplay.cpu_limit=32 --ResourceUseDisplay.mem_limit=128000000000 >&2

But the disadvantages of this are obvious:

  1. Password is hardcoded (and its hash is exposed), so I need different docker images for different users
  2. I can't change anything without rebuilding docker image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant