Support for ARM architecture?

[kingjerod]

I'm not sure if this is a problem with ARM. I'm trying to install gifsicle on a Docker image running an an ARM (AWS t4g server). This is the Dockerfile image:

arm64v8/node:14.18.1

Server is running Amazon Linux 2. Public AMI is ami-02b1ab520319e74be

The error I get when adding the package:

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/app/node_modules/gifsicle/\u0001�4@@��'"

Edit:
I'm able to install it in the image using apt-get install and then run it on the command line. Just something about this NPM package.

[amiller-gh]

Hey @kingjerod – you ever get this figured out? It appears the module adds a junk file with unicode in the name at some point during the pre/post install...

[kingjerod]

I did not. I ended up just removing the package from my app. I did not find a suitable replacement either.

[devanshmshah]

Hey, @kingjerod @amiller-gh - Were you able to figure this out? The junk file is still there.

[rbruckheimer]

I'm having different issues, using imagemin 8.0.0 on Ubuntu Linux 20.04 on M1 architecture.
Getting errors during install, leaving behiind a gifsicle directory which is not empty.
Install errors:
at /home/notroot/git/stx/node_modules/bin-build/node_modules/execa/index.js:231:11
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async /home/notroot/git/stx/node_modules/optipng-bin/lib/install.js:17:4
npm WARN rollback Rolling back gifsicle@5.3.0 failed (this is probably harmless): ENOTEMPTY: directory not empty, rmdir '/home/notroot/git/stx/node_modules/imagemin-gifsicle/node_modules/gifsicle'
npm WARN rollback Rolling back imagemin-gifsicle@7.0.0 failed (this is probably harmless): ENOTEMPTY: directory not empty, rmdir '/home/notroot/git/stx/node_modules/imagemin-gifsicle/node_modules/gifsicle'

Installing again without first removing node_modules folder gives this error:
npm WARN prepare removing existing node_modules/ before installation
npm ERR! code ENOTEMPTY
npm ERR! syscall rmdir
npm ERR! path /home/notroot/git/stx/node_modules/imagemin-gifsicle/node_modules/gifsicle
npm ERR! errno -39
npm ERR! ENOTEMPTY: directory not empty, rmdir '/home/notroot/git/stx/node_modules/imagemin-gifsicle/node_modules/gifsicle'

npm ERR! A complete log of this run can be found in:
npm ERR! /home/notroot/.npm/_logs/2022-04-28T03_32_14_819Z-debug.log

[gilbsgilbs]

TL;DR;

• sudo ln -fs /bin/bash /bin/sh should prevent the dangling file from being created and fix the build in all cases (but may have unintended side-effects)
• don't use yarn 1. yarn 2+ will still create the dangling file (like npm), but at least won't crash because of it.
• this project shouldn't try to execute x64 binary on arm64 arch in the first place, so there's probably also a bug to fix here (or in one of the libraries). I opened fix crash at installation on linux platforms with non-intel archs #145 which is more a workaround than an actual fix. ARM64 builds should also be provided along with this package.

---

After digging a bit, I found that the weird empty file is created at this point. Because the file name is not a valid utf-8 string it makes yarn 1 crash at linking step, but yarn 2 (berry) seems to be fine with it and can proceed normally (like npm).

The gifsicle --version command is used to detect at installation whether the provided binary works on the current platform (to rebuild gifsicle if it isn't). It passes through a bunch of libraries (bin-wrapper, bin-check and execa) and eventually execa invokes the command through node's child_process.spawn which ends up invoking unix's execvp in a subprocess. At this point, as the x86_64 ELF file is not executable on an ARM64 architecture, it gets executed by as some shell script (sic). As you can see the ELF file starts with something that looks like a shell redirection (notice the > character):

$ curl -Lo - 'https://github.com/imagemin/gifsicle-bin/raw/da45b6d92e6c8b08930bbfa2c38a41672e137e13/vendor/linux/x64/gifsicle' 2> /dev/null | xxd | head -3
00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000  .ELF............
00000010: 0200 3e00 0100 0000 f034 4000 0000 0000  ..>......4@.....
00000020: 4000 0000 0000 0000 f0eb 0a00 0000 0000  @...............

So this creates the file named after the weird random bytes following the > character (and explains the few errors we get when running yarn or npm in --verbose mode).

And indeed, if you run:

require("child_process").spawnSync("/path/to/gifsicle_x86_64", []).stderr.toString()

on Ubuntu/Debian and aarch64, you'll see the dangling file created.

Apparently, falling back to a shell in case exec* errors with ENOEXEC is what's specified in POSIX and what's implemented in glibc. The shell however may refuse to run such file, but there is no guarantee. bash does this well and refuses to execute the file, but dash (which is the default shell on debian based distros) doesn't and happily starts to execute it, behaving in an undefined way. Indeed, you can just run dash ./gifsicle_x86_64 (even on a x86_64 system) to reproduce the creation of the dangling file. This means that changing the default shell to bash (sudo ln -fs /bin/bash /bin/sh) or another POSIX-compliant shell that refuses to execute non-regular files (as suggested in latest POSIX) would just prevent the creation of this file and therefore fix the build everywhere.

Also, it seems that bin-wrapper uses os-file-obj to match available executables with the current platform and architecture, and should therefore have prevented the execution of the file in the first place. Unfortunately, os-file-obj still has undefined behavior with non-intel architecture and wrongfully returns the x64 binary as compatible, which leads to the execution of the file. To make matters worse, os-file-obj looks rather unmaintained (just like bin-wrapper and bin-check), so I'm not expecting a fix in there…

I think the simplest workaround for the sake of this project would be to throw an error before calling gifsicle --version if the platform is linux and the architecture is not x86_64, so that a build is triggered without node creating the cataclysmic empty file. I'll probably draft a PR with this solution as it seems quite easy and unintrusive to implement. Update: Implemented in #145.

It also might be worth discussing the usage of bin-wrapper and bin-check libraries as they seem unmaintained, dubious security-wise (downloading binary files from the internet when it's already available locally is not a good idea), and don't support modern architectures. At least providing binaries for a few other architectures and not downloading them from GitHub when they're available locally would be helpful.

By the way, other projects from the imagemin workspace are affected by this same bug (at least optipng, pngquant and cwebp, maybe others).

Update:

I've opened the following PRs to fix this issue on various imagemin projects:

• fix crash at installation on linux platforms with non-intel archs #145
• fix crash at installation on linux platforms with non-intel archs cwebp-bin#60
• fix crash at installation on linux platforms with non-intel archs optipng-bin#126
• fix crash at installation on linux platforms with non-intel archs pngquant-bin#140

[kingjerod]

I don't know if this is the same issue, I do see the PR mentioning this thread in the pngquant-bin repo. I was getting a similar error while trying to install pngquant using yarn:

"ENOENT: no such file or directory, lstat '/home/jerod/project/node_modules/pngquant-bin/\u0001�'"

When I switched from yarn to npm, it just worked. So try switching to npm!

[gilbsgilbs]

@kingjerod yes, it is the same issue, and I also submitted a PR here: imagemin/pngquant-bin#140