Build is broken and Node.js simple server service fails for Siemens SIMATIC IOT2000

[gghidini]

Scenario

I'm testing the simple server node application at https://github.com/balena-io-projects/simple-server-node on a Siemens SIMATIC IOT2040 (IOT2000 series) provisioned on balena.io.

I've been following the getting started guide at https://www.balena.io/docs/learn/getting-started/iot2000/nodejs/#adding-your-first-device.

I managed to provision the device successfully.

Problem(s) and some quick fixes

When I build (and deploy) the simple server node container on Balena I observe the following error:

$ git push balena master
Total 0 (delta 0), reused 0 (delta 0)

[Info]     Starting build for first-simatic-iot-2040-app, user [OMIT]
[Info]     Dashboard link: https://dashboard.balena-cloud.com/apps/1493325/devices
[Info]     Building on x64_01
[Info]     Pulling previous images for caching purposes...
[Success]  Successfully pulled cache images
[main]     Step 1/7 : FROM balenalib/iot2000-node:10-stretch-run
[main]     manifest for balenalib/iot2000-node:10-stretch-run not found
[Error]    Some services failed to build:
[Error]      Service: main
[Error]        Error: manifest for balenalib/iot2000-node:10-stretch-run not found
[Error]    Not deploying release.

remote: error: hook declined to update refs/heads/master
To [OMIT]@git.balena-cloud.com:[OMIT]/first-simatic-iot-2040-app.git
 ! [remote rejected] master -> master (hook declined)
error: failed to push some refs to '[OMIT]@git.balena-cloud.com:[OMIT]/first-simatic-iot-2040-app.git'

The build fails because tag 10-stretch-run does not exist for image balenalib/iot2000-node.

Furthermore, when I swap from 10-stretch-run to an existing tag such as 6.17.1-buster-build-20190503, the build process fails with the following errors:

$ git push balena master --force
Total 0 (delta 0), reused 0 (delta 0)

[Info]     Starting build for first-simatic-iot-2040-app, user [OMIT]
[Info]     Dashboard link: https://dashboard.balena-cloud.com/apps/1493325/devices
[Info]     Building on x64_01
[Info]     Pulling previous images for caching purposes...
[Success]  Successfully pulled cache images
[main]     Step 1/7 : FROM balenalib/iot2000-node:6.17.1-buster-build-20190503
[main]      ---> 0d75afc627c7
[main]     Step 2/7 : WORKDIR /usr/src/app
[main]      ---> d235a409ac1f
[main]     Removing intermediate container 9396d61891e8
[main]     Step 3/7 : COPY package.json package.json
[main]      ---> 633641444b59
[main]     Removing intermediate container 0e72391c6b4e
[main]     Step 4/7 : RUN JOBS=MAX npm install --production --unsafe-perm && npm cache verify && rm -rf /tmp/*
[main]      ---> Running in 8270a9ac8844
[main]     simple-server-node@1.0.0 /usr/src/app
[main]     ├── ejs@2.6.2
[main]     └─┬ express@4.17.1
[main]       ├─┬ accepts@1.3.7
[main]       │ ├─┬ mime-types@2.1.24
[main]       │ │ └── mime-db@1.40.0
[main]       │ └── negotiator@0.6.2
[main]       ├── array-flatten@1.1.1
[main]       ├─┬ body-parser@1.19.0
[main]       │ ├── bytes@3.1.0
[main]       │ ├─┬ http-errors@1.7.2
[main]       │ │ ├── inherits@2.0.3
[main]       │ │ └── toidentifier@1.0.0
[main]       │ ├─┬ iconv-lite@0.4.24
[main]       │ │ └── safer-buffer@2.1.2
[main]       │ └── raw-body@2.4.0
[main]       ├── content-disposition@0.5.3
[main]       ├── content-type@1.0.4
[main]       ├── cookie@0.4.0
[main]       ├── cookie-signature@1.0.6
[main]       ├─┬ debug@2.6.9
[main]       │ └── ms@2.0.0
[main]       ├── depd@1.1.2
[main]       ├── encodeurl@1.0.2
[main]       ├── escape-html@1.0.3
[main]       ├── etag@1.8.1
[main]       ├─┬ finalhandler@1.1.2
[main]       │ └── unpipe@1.0.0
[main]       ├── fresh@0.5.2
[main]       ├── merge-descriptors@1.0.1
[main]       ├── methods@1.1.2
[main]       ├─┬ on-finished@2.3.0
[main]       │ └── ee-first@1.1.1
[main]       ├── parseurl@1.3.3
[main]       ├── path-to-regexp@0.1.7
[main]       ├─┬ proxy-addr@2.0.5
[main]       │ ├── forwarded@0.1.2
[main]       │ └── ipaddr.js@1.9.0
[main]       ├── qs@6.7.0
[main]       ├── range-parser@1.2.1
[main]       ├── safe-buffer@5.1.2
[main]       ├─┬ send@0.17.1
[main]       │ ├── destroy@1.0.4
[main]       │ ├── mime@1.6.0
[main]       │ └── ms@2.1.1
[main]       ├── serve-static@1.14.1
[main]       ├── setprototypeof@1.1.1
[main]       ├── statuses@1.5.0
[main]       ├─┬ type-is@1.6.18
[main]       │ └── media-typer@0.3.0
[main]       ├── utils-merge@1.0.1
[main]       └── vary@1.1.2
[main]     npm
[main]      ERR! Usage: npm cache add <tarball file>
[main]     npm
[main]     ERR! npm cache add <folder>
[main]     npm ERR!
[main]      npm cache add <tarball url>
[main]     npm
[main]      
[main]     ERR! npm cache add <git url>
[main]     npm ERR! npm cache add <name>@<version>
[main]     npm
[main]      ERR! npm cache ls [<path>]
[main]     npm ERR! npm cache clean [<pkg>[@<version>]]
[main]     
[main]     Removing intermediate container 8270a9ac8844
[Info]     Uploading images
[main]     The command '/bin/sh -c JOBS=MAX npm install --production --unsafe-perm && npm cache verify && rm -rf /tmp/*' returned a non-zero code: 1
[Success]  Successfully uploaded images
[Error]    Some services failed to build:
[Error]      Service: main
[Error]        Error: The command '/bin/sh -c JOBS=MAX npm install --production --unsafe-perm && npm cache verify && rm -rf /tmp/*' returned a non-zero code: 1
[Error]    Not deploying release.

remote: error: hook declined to update refs/heads/master
To [OMIT]@git.balena-cloud.com:[OMIT]/first-simatic-iot-2040-app.git
 ! [remote rejected] master -> master (hook declined)
error: failed to push some refs to '[OMIT]@git.balena-cloud.com:[OMIT]/first-simatic-iot-2040-app.git'

I can workaround this error by running a plain npm install.

Following this configuration, the image builds on Balena Cloud, and is deployed to the Siemens SIMATIC IOT2040 device:

$ git push balena master
Counting objects: 175, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (97/97), done.
Writing objects: 100% (175/175), 403.25 KiB | 0 bytes/s, done.
Total 175 (delta 88), reused 143 (delta 70)
remote: Resolving deltas: 100% (88/88), done.

[Info]     Starting build for first-simatic-iot-2040-app, user [OMIT]
[Info]     Dashboard link: https://dashboard.balena-cloud.com/apps/1493325/devices
[Info]     Building on x64_01
[Info]     Pulling previous images for caching purposes...
[Success]  Successfully pulled cache images
[main]     Step 1/7 : FROM balenalib/iot2000-node:6.17.1-buster-build-20190503
[main]      ---> 0d75afc627c7
[main]     Step 2/7 : WORKDIR /usr/src/app
[main]     Using cache
[main]      ---> 78eed34ca76e
[main]     Step 3/7 : COPY package.json package.json
[main]     Using cache
[main]      ---> e4457d5c4b2f
[main]     Step 4/7 : RUN npm install
[main]      ---> Running in c1df313d8db0
[main]     simple-server-node@1.0.0 /usr/src/app
[main]     ├── ejs@2.6.2
[main]     └─┬ express@4.17.1
[main]       ├─┬ accepts@1.3.7
[main]       │ ├─┬ mime-types@2.1.24
[main]       │ │ └── mime-db@1.40.0
[main]       │ └── negotiator@0.6.2
[main]       ├── array-flatten@1.1.1
[main]       ├─┬ body-parser@1.19.0
[main]       │ ├── bytes@3.1.0
[main]       │ ├─┬ http-errors@1.7.2
[main]       │ │ ├── inherits@2.0.3
[main]       │ │ └── toidentifier@1.0.0
[main]       │ ├─┬ iconv-lite@0.4.24
[main]       │ │ └── safer-buffer@2.1.2
[main]       │ └── raw-body@2.4.0
[main]       ├── content-disposition@0.5.3
[main]       ├── content-type@1.0.4
[main]       ├── cookie@0.4.0
[main]       ├── cookie-signature@1.0.6
[main]       ├─┬ debug@2.6.9
[main]       │ └── ms@2.0.0
[main]       ├── depd@1.1.2
[main]       ├── encodeurl@1.0.2
[main]       ├── escape-html@1.0.3
[main]       ├── etag@1.8.1
[main]       ├─┬ finalhandler@1.1.2
[main]       │ └── unpipe@1.0.0
[main]       ├── fresh@0.5.2
[main]       ├── merge-descriptors@1.0.1
[main]       ├── methods@1.1.2
[main]       ├─┬ on-finished@2.3.0
[main]       │ └── ee-first@1.1.1
[main]       ├── parseurl@1.3.3
[main]       ├── path-to-regexp@0.1.7
[main]       ├─┬ proxy-addr@2.0.5
[main]       │ ├── forwarded@0.1.2
[main]       │ └── ipaddr.js@1.9.0
[main]       ├── qs@6.7.0
[main]       ├── range-parser@1.2.1
[main]       ├── safe-buffer@5.1.2
[main]       ├─┬ send@0.17.1
[main]       │ ├── destroy@1.0.4
[main]       │ ├── mime@1.6.0
[main]       │ └── ms@2.1.1
[main]       ├── serve-static@1.14.1
[main]       ├── setprototypeof@1.1.1
[main]       ├── statuses@1.5.0
[main]       ├─┬ type-is@1.6.18
[main]       │ └── media-typer@0.3.0
[main]       ├── utils-merge@1.0.1
[main]       └── vary@1.1.2
[main]      ---> aa31e13ee8ba
[main]     Removing intermediate container c1df313d8db0
[main]     Step 5/7 : COPY . ./
[main]      ---> 123a4ede8b03
[main]     Removing intermediate container c407ee15933e
[main]     Step 6/7 : ENV UDEV 1
[main]      ---> Running in b472a4a4d9ae
[main]      ---> 74c2ba822cb8
[main]     Removing intermediate container b472a4a4d9ae
[main]     Step 7/7 : CMD npm start
[main]      ---> Running in 2c34f2586be7
[main]      ---> 99c710e9402b
[main]     Removing intermediate container 2c34f2586be7
[main]     Successfully built 99c710e9402b
[Info]     Uploading images
[Success]  Successfully uploaded images
[Success]  Release successfully created!
[Info]     Release: 70088570351f57da91ae454a2670f45b3f2a52ee (id: 994674)
[Info]     ┌─────────┬────────────┬────────────┐
[Info]     │ Service │ Image Size │ Build Time │
[Info]     ├─────────┼────────────┼────────────┤
[Info]     │ main    │ 705.74 MB  │ 7 seconds  │
[Info]     └─────────┴────────────┴────────────┘
[Info]     Build finished in 14 seconds
			    \
			     \
			      \\
			       \\
			        >\/7
			    _.-(6'  \
			   (=___._/` \
			        )  \ |
			       /   / |
			      /    > /
			     j    < _\
			 _.-' :      ``.
			 \ r=._\        `.
			<`\\_  \         .`-.
			 \ r-7  `-. ._  ' .  `\
			  \`,      `-.`7  7)   )
			   \/         \|  \'  / `-._
			              ||    .'
			               \\  (
			                >\  >
			            ,.-' >.'
			           <.'_.''
			             <'

To [OMIT]@git.balena-cloud.com:[OMIT]/first-simatic-iot-2040-app.git
 * [new branch]      master -> master

However, the service keeps failing on the device, so that the simple server node Web page is never available at the public URL:

$ curl https://[OMIT].balena-devices.com/
<!DOCTYPE html><html><head><meta charset="utf8">
<!-- [...] -->
<p><code>tunneling socket could not be established: socket hang up</code></p>
<p>One possible reason is because nothing is listening on port 80 on the device.</p><p><a class="btn btn-block btn-info" href="">RETRY</a></p></div><div class="footer"><p>Device Public URLs is a <a href="https://balena.io"><strong>balena.io</strong></a>
feature that provides individual URLs for balena.io-connected devices.
The feature is opt-in: no devices are exposed without the application owner's permission.</p>
<p><strong>balena.io</strong> does not serve anything on this URL, we just
proxy all the requests to whatever is listening on the device. This means
that the creator of the application is responsible for running a web-server

When I attempt to securely log in to the service from Balena Cloud, the operation fails because the service is not running:

Connecting to 58aca91762244c0a019d162aa3e0460e...
Spawning shell...
Error response from daemon: Container cc2cd177ca7ee3ddf67597668af84835718909afcf76fb8638f2ed1e1f4db129 is restarting, wait until the container is running
Error response from daemon: Container cc2cd177ca7ee3ddf67597668af84835718909afcf76fb8638f2ed1e1f4db129 is restarting, wait until the container is running
SSH session disconnected
SSH reconnecting...
Spawning shell...
Error response from daemon: Container cc2cd177ca7ee3ddf67597668af84835718909afcf76fb8638f2ed1e1f4db129 is restarting, wait until the container is running
Error response from daemon: Container cc2cd177ca7ee3ddf67597668af84835718909afcf76fb8638f2ed1e1f4db129 is restarting, wait until the container is running
SSH session disconnected

I have not investigated this specific problem further.

Proposed solution(s)

The straightforward solution is to align the base images for all architectures.

However, being new to Balena, I'm not sure why there is no Node v. 10 base image for Siemens SIMATIC IOT2000.

Alternatively, different branches in the git repo could be created for different architectures, so that each can be config w/ relevant base image tag.

I don't have a proposed solution for the failing service.

Next actions

Once we get a first round of discussion on these problems (missing base image, failing npm command, and service unavailable), we can split them accordingly into three separate issues.

I'll be glad to discuss this further, and help implement an agreed-upon fix.