Install instructions fail confusingly if curl fails

[SamStephens]

Describe your bug

Install instructions for all platforms are similar to these for Ubuntu:

curl -fsSL https://deb.nodesource.com/setup_21.x | sudo -E bash - &&\
sudo apt-get install -y nodejs

From the README. However, as per the discussion on #1790, if curl fails, execution continues to the sudo apt-get install -y nodejs, so NodeJS is still installed - but the nodejs package your Linux distribution provides, not the nodejs package you are distributing.

Distribution Information:

• OS: Applies to all
• Version: Applies to all

Node Version:

• Node: Applies to all

To Reproduce

You can see this using the following Dockerfile. The base image python:3.11-slim-bookworm does not include curl.

FROM python:3.11-slim-bookworm

RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
RUN node -v
RUN npm -v

Instead of failing on the curl line, we get to the npm line, which then fails because the Debian nodejs package appears not to include npm.

Expected behavior

If curl is unavailable, or otherwise has an error (transient 500 or whatever), execution should stop at that point.

Additional context

When curl is unavailable,

curl -fsSL https://deb.nodesource.com/setup_20.x | bash -

Has an exit code of 0,

curl -fsSL https://deb.nodesource.com/setup_20.x

Has an exit code of 127.

I'm not that deep on shell internals, but it appears the exit code is the zero from the bash that is the pipe destination, not the curl command pipe source with it's 127 exit code.

[riosje]

Hi @SamStephens I tried to replicate the issue but was not able to.
I used the dockerfile example you posted, and it fails as expected because the curl does not exist.
Could you share more details about your setup?

FROM python:3.11-slim-bookworm

RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
RUN node -v
RUN npm -v

(base) ➜  Docker docker build -t debug .
[+] Building 0.9s (5/7)                                                                                     docker:desktop-linux
 => [internal] load build definition from dockerfile                                                                        0.0s
 => => transferring dockerfile: 185B                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.11-slim-bookworm                                                0.5s
 => [internal] load .dockerignore                                                                                           0.0s
 => => transferring context: 2B                                                                                             0.0s
 => CACHED [1/4] FROM docker.io/library/python:3.11-slim-bookworm@sha256:3800945e7ed50341ba8af48f449515c0a4e845277d56008c1  0.0s
 => => resolve docker.io/library/python:3.11-slim-bookworm@sha256:3800945e7ed50341ba8af48f449515c0a4e845277d56008c15bd84d5  0.0s
 => ERROR [2/4] RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs                  0.3s
------
 > [2/4] RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs:
0.163 /bin/sh: 1: curl: not found
0.273 Reading package lists...
0.289 Building dependency tree...
0.289 Reading state information...
0.290 E: Unable to locate package nodejs
------
dockerfile:3
--------------------
   1 |     FROM python:3.11-slim-bookworm
   2 |
   3 | >>> RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
   4 |     RUN node -v
   5 |     RUN npm -v
--------------------
ERROR: failed to solve: process "/bin/sh -c curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs" did not complete successfully: exit code: 100

[SamStephens]

@riosje apologies for the confusion here, this is my fault for editing the Dockerfile without retesting. I removed RUN apt-get update --fix-missing && apt-get upgrade -y && apt-get dist-upgrade -y from my original reproduction to speed it up, but it looks like the Debian version of the nodejs package is only present after that line is executed.

However, note the error you're seeing there, Unable to locate package nodejs. This is because curl -fsSL https://deb.nodesource.com/setup_20.x | bash - is finishing with a return code of 0, and apt-get install -y nodejs is being executed and failing.

You can see this more clearly by adding back the update line I removed, so that nodejs is present, but it's the Debian version, not the nodesource version.

FROM python:3.11-slim-bookworm

RUN apt-get update --fix-missing && apt-get upgrade -y && apt-get dist-upgrade -y

RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
RUN node -v
RUN npm -v

$ docker build -t temp-nodesource .
[+] Building 1.1s (8/8) FINISHED                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 269B                                                                               0.0s
 => [internal] load metadata for docker.io/library/python:3.11-slim-bookworm                                       0.8s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [1/5] FROM docker.io/library/python:3.11-slim-bookworm@sha256:3800945e7ed50341ba8af48f449515c0a4e845277d56008  0.0s
 => CACHED [2/5] RUN apt-get update --fix-missing && apt-get upgrade -y && apt-get dist-upgrade -y                 0.0s
 => CACHED [3/5] RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs        0.0s
 => CACHED [4/5] RUN node -v                                                                                       0.0s
 => ERROR [5/5] RUN npm -v                                                                                         0.3s
------
 > [5/5] RUN npm -v:
0.242 /bin/sh: 1: npm: not found
------
Dockerfile:7
--------------------
   5 |     RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
   6 |     RUN node -v
   7 | >>> RUN npm -v
   8 |
   9 |
--------------------
ERROR: failed to solve: process "/bin/sh -c npm -v" did not complete successfully: exit code: 127

[Mustafa1p]

The behavior you're experiencing indeed stems from how shell pipes (|) handle exit statuses. By default, a pipeline's exit status is that of the last command. In your case, even if curl fails (which should ideally halt the installation process), the script proceeds with the execution of subsequent commands because bash (the last command in the pipeline) exits with a status of 0 (success).

To address this issue, you can modify the script to ensure it stops executing if any command within a pipeline fails. This can be achieved by setting the pipefail option in your shell. When pipefail is set, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
Updated Script with pipefail

Here's how you can incorporate pipefail into your script:

dockerfile

FROM python:3.11-slim-bookworm

Ensure the script exits if any command in a pipeline fails

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

Attempt to download and execute the setup script; install Node.js

RUN apt-get update && apt-get install -y curl &&
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - &&
apt-get install -y nodejs

Verify installation

RUN node -v
RUN npm -v

Explanation:

SHELL ["/bin/bash", "-o", "pipefail", "-c"]: This line changes the default shell used for RUN commands to bash with the pipefail option enabled. This ensures that if any command in a pipeline fails, the entire pipeline fails, preventing execution of further commands.
apt-get update && apt-get install -y curl: Before attempting to run the curl command, this ensures that curl is installed. This is particularly necessary in slim or minimal base images where curl might not be available by default.

Additional Consideration:

To make your script more robust, you might also want to check the availability of critical commands like curl before attempting to use them, especially when working with minimal Docker images or environments where the availability of such commands cannot be taken for granted.
Conclusion:

By employing pipefail, you ensure that the shell script respects the exit status of all commands in a pipeline, providing a more reliable way to handle potential failures and prevent unintended behavior during the installation process.

[SamStephens]

@Mustafa1p I am aware of pipefail. I'm not asking for this guidance for myself. I'm wondering if the shell commands provided in the README can be made more robust.

[JesusPaz]

Hi @SamStephens

These instructions have been in the repo for over 5 years without changes. We understand that users have different levels of experience and knowledge, so we decided to update our instructions to make them more explanatory and step-by-step. I hope these changes meet your expectations. If you would like to suggest any further modifications, please let me know, and we can create a pull request to update these files.

#1803

[SamStephens]

@JesusPaz thanks for these changes.

I want to be clear here, this isn't just about experience and knowledge. Regardless of your experience as a developer, if you used the instructions including curl -fsSL https://deb.nodesource.com/setup_20.x | bash - in a script, and the curl command failed because of a transient 500 or some such, you'd get the extremely confusing behavior of your script succeeding, but installing the OS distribution of nodejs, rather than the nodesource distribution.

[riosje]

Hi @SamStephens would be much more easier for every one if we remove the curl command from our instructions and just put something like:

Download and execute the following bash script.

https://deb.nodesource.com/setup_20.x

At this point the user will be more conscious that the script must be downloaded and executed.
Becase sometimes it feels like we're being responsible for the most minimal en-user mistakes.

[SamStephens]

@riosje my point here is there's no end user mistake involved. The previously recommended instructions curl -fsSL https://deb.nodesource.com/setup_20.x | bash - were not robust and would fail confusingly if the curl request failed for reasons beyond the end users control.

Statements like

users have different levels of experience and knowledge

and

Because (sic) sometimes it feels like we're being responsible for the most minimal end-user (sic) mistakes.

Are not helpful. I'm trying to help make the instructions provided more robust in the face of transient failures. It's kinda toxic that members of your team seem determined to make this about user error.