Cannot extract OTP release tarball during Kaniko build

[vassilevsky]

The bug

This bug is very interesting. I just had to share :)

Usually OTP releases on GitHub are packed by runner:docker (1001:127):

$ tar -tvf otp_src_25.3.2.8.tar | head -n 3
drwxr-xr-x  0 runner docker      0 Dec 18 19:34 otp_src_25.3.2.8/
-rw-r--r--  0 runner docker    170 Dec 15 22:23 otp_src_25.3.2.8/erlang_ls.config
drwxr-xr-x  0 runner docker      0 Dec 15 22:23 otp_src_25.3.2.8/plt/

$ tar -tv --numeric-owner -f otp_src_25.3.2.8.tar | head -n 3
drwxr-xr-x  0 1001   127         0 Dec 18 19:34 otp_src_25.3.2.8/
-rw-r--r--  0 1001   127       170 Dec 15 22:23 otp_src_25.3.2.8/erlang_ls.config
drwxr-xr-x  0 1001   127         0 Dec 15 22:23 otp_src_25.3.2.8/plt/

But a recent release was packed by a different, strange user:

$ tar -tvf otp_src_25.3.2.9.tar | head -n 3
drwxr-xr-x  0 eluklar eusers      0 Feb  8 18:42 otp_src_25.3.2.9/
drwxr-xr-x  0 eluklar eusers      0 Feb  7 18:01 otp_src_25.3.2.9/xcomp/
-rw-r--r--  0 eluklar eusers  10873 Feb  7 18:01 otp_src_25.3.2.9/xcomp/erl-xcomp-powerpc-dso-linux-gnu.conf

$ tar -tv --numeric-owner -f otp_src_25.3.2.9.tar | head -n 3
drwxr-xr-x  0 107466  64000       0 Feb  8 18:42 otp_src_25.3.2.9/
drwxr-xr-x  0 107466  64000       0 Feb  7 18:01 otp_src_25.3.2.9/xcomp/
-rw-r--r--  0 107466  64000   10873 Feb  7 18:01 otp_src_25.3.2.9/xcomp/erl-xcomp-powerpc-dso-linux-gnu.conf

Would you look at this UID.

I build an image from the following Dockerfile, nothing fancy:

FROM centos:7

# ASDF

RUN yum install -y git
ENV ASDF_DIR=/opt/asdf
RUN git clone https://github.com/asdf-vm/asdf.git $ASDF_DIR
ENV PATH=$ASDF_DIR/shims:$ASDF_DIR/bin:$PATH
ENV ASDF_DATA_DIR=$ASDF_DIR

# Erlang

RUN yum install -y make autoconf gcc ncurses-devel openssl-devel
RUN asdf plugin add erlang
RUN asdf install erlang 24.3.4.16
RUN asdf install erlang 25.3.2.9
RUN asdf install erlang 26.2.2

I run Kaniko (gcr.io/kaniko-project/executor:debug) to build the image.

It builds OTP 24.3.4.16 just fine. But OTP 25.3.2.9 fails with this:

tar: otp_src_25.3.2.9/xcomp/erl-xcomp-powerpc-dso-linux-gnu.conf: Cannot change ownership to uid 107466, gid 64000: Invalid argument
tar: otp_src_25.3.2.9/xcomp/erl-xcomp-arm64-android.conf: Cannot change ownership to uid 107466, gid 64000: Invalid argument
tar: otp_src_25.3.2.9/xcomp/erl-xcomp-powerpc64-bgq-linux.conf: Cannot change ownership to uid 107466, gid 64000: Invalid argument
...

It says that for every file in the archive.

Tar has options:

--same-owner
	try extracting files with the same ownership as exists in the archive (default for superuser)
-p, --preserve-permissions, --same-permissions
	extract information about file permissions (default for superuser)

--no-same-owner
	extract files as yourself (default for ordinary users)
--no-same-permissions
	apply the user's umask when extracting permissions from the archive (default for ordinary users)

So I guess that Kaniko runs builds as superuser by default, tries to set UID and GID from the archive, and fails spectacularly :)

Software versions

All the software is the latest one. Kerl is downloaded by asdf-erlang automatically.

How to replicate

Build provided Dockerfile with Kaniko.

Expected behaviour

OTP 25.3.2.9 is installed.

Additional context

I think that bothering OTP team with this issue is not necessary.

[paulo-ferraz-oliveira]

👋 @vassilevsky.

I'm not sure how kerl is involved in this, but maybe I'm missing a detail here.

Note: eluklar is potentially @garazdawi, and he might have a better explanation than me as to why the packaging was so.

In any case, kerl can build 25.3.2.9, as per this repo's CI.

What would you suggest we do? Or what is the exact issue we should look at?

[garazdawi]

eluklar is indeed me. I think github actions failed for 25.3.2.9 so I had to build the pre-built source manually which is why it shows up like that.

[vassilevsky]

Nice to meet you, Paulo :)

For now I have solved this issue by pointing Kerl to my fork of OTP. My fork does not have releases, so Kerl resorts to downloading tags. In the archives made by GitHub, files are owned by root:

$ tar -tvf otp-OTP-25.3.2.9.tar | head -n 3
drwxrwxr-x  0 root   root        0 Feb  7 18:01 otp-OTP-25.3.2.9/
drwxrwxr-x  0 root   root        0 Feb  7 18:01 otp-OTP-25.3.2.9/.devcontainer/
-rw-rw-r--  0 root   root     1188 Feb  7 18:01 otp-OTP-25.3.2.9/.devcontainer/devcontainer.json

$ tar -tv --numeric-owner -f otp-OTP-25.3.2.9.tar | head -n 3
drwxrwxr-x  0 0      0           0 Feb  7 18:01 otp-OTP-25.3.2.9/
drwxrwxr-x  0 0      0           0 Feb  7 18:01 otp-OTP-25.3.2.9/.devcontainer/
-rw-rw-r--  0 0      0        1188 Feb  7 18:01 otp-OTP-25.3.2.9/.devcontainer/devcontainer.json

They are extracted without problems and the build succeeds :)

Very nice to meet you, Lukas :) Thank you for making this effort of uploading OTP archives to GitHub releases. I found your issue #351 where you announced it and it led me to this discovery. I did not want to bother you at all with this, but oh well.

I think that one possible thing that Kerl might do to prevent errors like this in the future is to always run tar --no-same-owner --no-same-permissions. This is the default for non-root users, and it could very well be enabled for root user as well. I see no reason to preserve ownership when extracting any OTP archive, regardless of its source. File ownership inside the archive should not matter. I could make this change myself and send you a pull request, if you agree.

Or we might as well do nothing, because this is such a niche problem, and next OTP archives will most likely be made by CI anyway :)

[paulo-ferraz-oliveira]

File ownership inside the archive should not matter. I could make this change myself and send you a pull request, if you agree

I think it's a reasonable change. Just be sure to make it POSIX compliant, since kerl is usable under Linux and macOS, at least. CI should be enough to test it, but one never knows.