Batect v0.80.0 unable to start on Alpine Linux

[Tobytee17]

Describe the bug

When running any Batect task as part of our GitLab-CI build pipeline after upgrading to Batect v0.80.0, we run into the following issue:

$ ./batect docs
Downloading Batect version 0.80.0 from https://updates.batect.dev/v1/files/0.80.0/batect-0.80.0.jar...
#=#=#                                                                         
#=#=-#  #                                                                     
-#O#- #   #                                                                   
######################################################################## 100.0%
Could not establish connection to Docker daemon: Error relocating /tmp/batect-docker-client171999[42](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L42)506294148[44](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L44)1/libdockerclientwrapper.so: (null): initial-exec TLS resolves to dynamic definition in /tmp/batect-docker-client171999425062941[48](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L48)441/libdockerclientwrapper.so
Library names
[dockerclientwrapper]
Search paths:
[/tmp/batect-docker-client17199942[50](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L50)6294148441, /usr/lib/jvm/java-11-openjdk/lib/server, /usr/lib/jvm/java-11-openjdk/lib, /usr/lib/jvm/java-11-openjdk/../lib, /usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib, /usr/local/lib]
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: .artifacts: no matching files             
ERROR: No files to upload                          
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

When reverting back to earlier versions of Batect (for example v0.78.0) this issue goes away:

$ ./batect docs
Downloading Batect version 0.78.0 from https://updates.batect.dev/v1/files/0.78.0/batect-0.78.0.jar...
#=#=#                                                                         
##O#-#                                                                        
######################################################################## 100.0%
Running docs...
Pulling asciidoctor/docker-asciidoctor...
Pulled asciidoctor/docker-asciidoctor.
Running " mkdir -p public && asciidoctor --require asciidoctor-diagram --backend xhtml --out-file public/index.html README.adoc " in asciidoctor...
asciidoctor: WARNING: utils/rules/README.adoc: line 1: section title out of sequence: expected level 2, got level 3
asciidoctor: WARNING: utils/rules/README.adoc: line 18: section title out of sequence: expected level 2, got level 3
asciidoctor: WARNING: utils/rules/README.adoc: line 54: section title out of sequence: expected level 2, got level 3
Cleaning up...
docs finished with exit code 0 in 33s.

Our GitLab runners use the Kubernetes executor to schedule/run builds, coupled with the Docker in Docker service (docker:20.10.8-dind) for spinning up containers as part of our build, and use Alpine v3.14.0 as the base image for the sub agents/runners.

Reproduction repo

No response

Steps to reproduce

1. Create or upgrade a project to use the Batect v0.80.0 binary.
2. Create a basic .gitlab-ci.yml file to run a Batect task (doesn't really matter what the task does, the failure occurs at the start of the execution when trying to establish a connection with the Docker Daemon).
3. Ensure that the GitLab runner's being used for scheduling are using either the Kubernetes or Docker runner executor (see https://docs.gitlab.com/runner/executors/) and are using an Alpine v3.14.0 base image for the sub-runners/agents, and ensure that the Docker service referenced in your build pipeline is using the Docker in Docker container docker:20.10.8-dind.
4. Run the GitLab-CI build pipeline.

I haven't had time to properly investigate yet, but because the issue appears to be around the initial connection with the Docker Daemon, it might be easier to recreate this issue by trying to run the task inside of a Alpine/Docker in Docker image (https://hub.docker.com/r/yobasystems/alpine-docker/ for example) to see if the issue occurs as well.

Expected behaviour

The task is run successfully (as it was prior to upgrading to Batect v0.80.0)

For example on v0.78.0:

$ ./batect docs
Downloading Batect version 0.78.0 from https://updates.batect.dev/v1/files/0.78.0/batect-0.78.0.jar...
#=#=#                                                                         
##O#-#                                                                        
######################################################################## 100.0%
Running docs...
Pulling asciidoctor/docker-asciidoctor...
Pulled asciidoctor/docker-asciidoctor.
Running " mkdir -p public && asciidoctor --require asciidoctor-diagram --backend xhtml --out-file public/index.html README.adoc " in asciidoctor...
asciidoctor: WARNING: utils/rules/README.adoc: line 1: section title out of sequence: expected level 2, got level 3
asciidoctor: WARNING: utils/rules/README.adoc: line 18: section title out of sequence: expected level 2, got level 3
asciidoctor: WARNING: utils/rules/README.adoc: line 54: section title out of sequence: expected level 2, got level 3
Cleaning up...
docs finished with exit code 0 in 33s.

Actual behaviour

The task fails to start as Batect appears to be unable to initiate a connection with the Docker Daemon, see the failure below when executing a task using Batect v0.80.0:

$ ./batect docs
Downloading Batect version 0.80.0 from https://updates.batect.dev/v1/files/0.80.0/batect-0.80.0.jar...
#=#=#                                                                         
#=#=-#  #                                                                     
-#O#- #   #                                                                   
######################################################################## 100.0%
Could not establish connection to Docker daemon: Error relocating /tmp/batect-docker-client171999[42](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L42)506294148[44](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L44)1/libdockerclientwrapper.so: (null): initial-exec TLS resolves to dynamic definition in /tmp/batect-docker-client171999425062941[48](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L48)441/libdockerclientwrapper.so
Library names
[dockerclientwrapper]
Search paths:
[/tmp/batect-docker-client17199942[50](https://gitlab.com/company/shared/build-resources/gitlab-pipelines/-/jobs/2956598982#L50)6294148441, /usr/lib/jvm/java-11-openjdk/lib/server, /usr/lib/jvm/java-11-openjdk/lib, /usr/lib/jvm/java-11-openjdk/../lib, /usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib, /usr/local/lib]
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: .artifacts: no matching files             
ERROR: No files to upload                          
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

Version information

Note: This output is from running "Batect --version" as part of the GitLab pipeline, so it should reflect the execution environment that the error is being thrown in. Locally on my work machine, Batect v0.80.0 works correctly.

Batect version:    0.80.0
Built:             2022-08-25 03:48:23 +0000
Built from commit: 9b9f8ac5a1c54e4f0f30e679ea810efe714651ca (commit date: 2022-08-25 12:57:47 +1000)
JVM version:       Alpine OpenJDK 64-Bit Server VM 11.0.11
OS version:        Linux 5.4.149-73.259.amzn2.x86_64 amd64 (Alpine Linux (alpine))
Docker version:    (could not get Docker version information because UnsatisfiedLinkError was thrown: Error relocating /tmp/batect-docker-client18154473160401624099/libdockerclientwrapper.so: (null): initial-exec TLS resolves to dynamic definition in /tmp/batect-docker-client18154473160401624099/libdockerclientwrapper.so
Library names
[dockerclientwrapper]
Search paths:
[/tmp/batect-docker-client18154473160401624099, /usr/lib/jvm/java-11-openjdk/lib/server, /usr/lib/jvm/java-11-openjdk/lib, /usr/lib/jvm/java-11-openjdk/../lib, /usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib, /usr/local/lib])
Git version:       2.32.0

Additional note, the Docker in Docker container used by the GitLab runner is docker:20.10.8-dind, so the version of Docker that should be displayed here is 20.10.8

Any other information

I am aware that the steps to recreate aren't exactly thorough (I wasn't sure how brief to keep it all considering the requirement of a GitLab pipeline/infrastructure for anyone else to recreate the exact environment/setup), so if there is anything additional information I can provide, or things I can try, please let me know.

[charleskorn]

Thanks for the detailed bug report @Tobytee17.

As you hinted at, this looks like an Alpine-specific issue, I can reproduce it without GitLab with the following command:

docker run --rm -it -v $(pwd):/code -w /code --platform linux/amd64 alpine:3.14.0 sh -c "apk add bash curl openjdk11-jre-headless && ./batect --version"

I think I know what the issue might be, but I'll have to investigate a little further to confirm.

[charleskorn]

Turns out this isn't what I thought it was. It's a bug in Golang, which was first reported late last year...

For now, I'd recommend sticking with Batect 0.79 or using a non-Alpine base image on GitLab.

[Tobytee17]

Ahh okay sweet as, thank you for the update and speedy investigation!

[stale]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. It will automatically be closed if no further activity occurs in the next seven days to enable maintainers to focus on the most important issues.
If this issue is still affecting you, please comment below within the next seven days.
Thank you for your contributions.