Missing python3 and cpu-feauture (optional) dependencies when doing docker deployment.

[corganfuzz]

Followed this guide an created 2 images with a separate frontend. I get 2 dependencies errors using the provided Dockerfiles (everything out of the box). Issue happens on Backend and frontend. Do I need to install python3 ? it's not specified on the instructions

[5/5] Building fresh packages...
warning Error running install script for optional dependency: "/app/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /app/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@5.1.0
gyp info using node@14.18.0 | linux | x64
gyp ERR! find Python
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON
gyp ERR! find Python checking if "python" can be used
gyp ERR! find Python - "python" is not in PATH or produced an error
gyp ERR! find Python checking if "python2" can be used
gyp ERR! find Python - "python2" is not in PATH or produced an error
gyp ERR! find Python checking if "python3" can be used
gyp ERR! find Python - "python3" is not in PATH or produced an error
gyp ERR! find Python
gyp ERR! find Python **********************************************************
gyp ERR! find Python You need to install the latest version of Python.
gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
gyp ERR! find Python you can try one of the following options:
gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
gyp ERR! find Python (accepted by both node-gyp and npm)
gyp ERR! find Python - Set the environment variable PYTHON
gyp ERR! find Python - Set the npm configuration variable python:
gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
gyp ERR! find Python For more information consult the documentation at:
gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
gyp ERR! find Python **********************************************************
gyp ERR! find Python
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Python installation to use
gyp ERR! stack at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
gyp ERR! stack at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)

Steps to Reproduce

1. npx @backstage/create-app (Node v14.15.5)
2. select postgresql as your db, I think it also happens on sqlite
3. CI is using ubuntu:latest but I also tested it on centos7
4. docker build . -f packages/backend/Dockerfile --tag backstage
5. Same error happens when you build the frontend

Backend Dockerfile

FROM node:14-buster-slim

WORKDIR /app
COPY yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz

RUN yarn install --frozen-lockfile --production --network-timeout 300000 && rm -rf "$(yarn cache dir)"

COPY packages/backend/dist/bundle.tar.gz app-config.yaml ./
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz

CMD ["node", "packages/backend", "--config", "app-config.yaml"]

Frontend Dockerfile

FROM node:14-buster AS build

RUN mkdir /app
COPY . /app
WORKDIR /app

RUN yarn install
RUN yarn workspace app build

FROM nginx:mainline

RUN apt-get update && apt-get -y install jq && rm -rf /var/lib/apt/lists/*
COPY --from=build /app/packages/app/dist /usr/share/nginx/html
COPY docker/default.conf.template /etc/nginx/templates/default.conf.template
COPY docker/inject-config.sh /docker-entrypoint.d/40-inject-config.sh
ENV PORT 80

Everything is built using create-app and deployment using docker with a separate image in the frontend.

Yarn.lock has the cpu-feature dependency

cpu-features@0.0.2:
  version "0.0.2"
  resolved "https://registry.yarnpkg.com/cpu-features/-/cpu-features-0.0.2.tgz#9f636156f1155fd04bdbaa028bb3c2fbef3cea7a"
  integrity sha512-/2yieBqvMcRj8McNzkycjW2v3OIUOibBfd2dLEJ0nWts8NobAxwiyw9phVNS6oDL8x8tz9F7uNVFEVpJncQpeA==
  dependencies:
    nan "^2.14.1"

[freben]

Hi! Yeah, if deps are now required which are built using node-gyp, you'll need python in the image. Would you mind trying to add a step for that and see if it resolves the problem?

[corganfuzz]

@freben I've solved it by adding this line within my 2 dockerfiles

RUN apt-get update && apt-get -y install python3 cmake g++

I honestly think this line should be included in the docs.

[corganfuzz]

Also, regarding cpu-features package. It says it's an optional dependency I can see it inside the main yarn.lock file so even if I delete it it still tries to install it

[5/5] Building fresh packages...
warning Error running install script for optional dependency: "/app/node_modules/cpu-features: Command failed.

any way I can remove it ? it is optional, right ? or is it not ?

[freben]

Hm I'm not sure if it's actually optional and what to do about that honestly.

In any case, yeah node-gyp will be needed. We'd appreciate if you added the python installation to the docs.

[Rugvip]

Generally node-gyp shouldn't need to be invoked as it's often just a fallback for when there aren't any published binaries, although that's not always the case.

To clarify the original issue, does this break the build or does it continue on to create a working image but you are concerned about the error?

It may be that we simply want to document that there will be a warning that can be ignored, but if you want to avoid it you can add the python installation to the docker build

[corganfuzz]

Mostly concerned. Nope it does not break the build. I see the error in every FROM node:14-buster-slim call. If I do a frontend separated from the backend with multi-stage I see it 3 times and it's nerve racking. specially if it's your first time trying to build backstage.

That's essentially what I want to know, can this be ignored ? why is cpu-features there? when I look for that dependency in this repo I cant find it. It seems to me that I can.

Also , for multi-stage build if I adding apt-get update + python3 adds an extra 300 MB to the backend container, so theres that

[Rugvip]

It's definitely safe to ignore. It's used by ssh2 to optimize the cipher ordering based on hardware features of the CPU. The ssh2 package is in turn a transitive dependency of testcontainers, which we use for testing towards databases running in docker, and with it being a devDependency they don't show up in the final build either.

I would avoid adding python as it's just gonna slow down the build process with no real benefit. It's a shame there's no way to opt-out of specific dependencies or anything like that (that I know of). Seems to be something we just need to live with tbh yarnpkg/yarn#3738

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alanhughes]

Since backstage 0.4.19, and swapping out sqlite3 for @vscode/sqlite3, this error now causes the build to fail, presumably because @vscode/sqlite3 is a required (rather than optional) dependency. Installing python, cmake and g++ resolves the issue - I believe this should be documented now that the build fails if these packages are not installed

[iverberk]

I'm running into this as well, out of the box installation results in a failed docker build:

#10 73.96 error /app/node_modules/@vscode/sqlite3: Command failed.

and then a lot of node-gyp output trying to find Python.

[bgutschke]

Since backstage 0.4.19, and swapping out sqlite3 for @vscode/sqlite3, this error now causes the build to fail, presumably because @vscode/sqlite3 is a required (rather than optional) dependency. Installing python, cmake and g++ resolves the issue - I believe this should be documented now that the build fails if these packages are not installed

Having the same problem. But installing python, cmake and g++ in the production image is not really an option, right? It fixes the issue of the dependencies, but then you have compilers in the image which could pose a security risk.

@backstage-team: Any chance that @vscode/sqlite3 will be defined as an optional dependency again?