Error pulling docker image from quay.io: "link: unpriv.link: unpriv.wrap target: no such file or directory"

[cfluta-latus]

Hello,

I've have been experiencing an issue pulling the docker image "recount-unify" from quay.io
Commands which worked for other recount images:

singularity pull docker://quay.io/benlangmead/recount-rs5:1.0.6
singularity pull docker://quay.io/broadsword/recount-pump:1.1.3

But unify produces this output:

singularity pull docker://quay.io/broadsword/recount-unify:1.1.1

INFO: Converting OCI blobs to SIF format INFO: Starting build... INFO: Fetching OCI image... 66.3KiB / 66.3KiB [======================================================================================================================] 100 % 0.0 b/s 0s 2.2MiB / 2.2MiB [========================================================================================================================] 100 % 0.0 b/s 0s 9.3MiB / 9.3MiB [========================================================================================================================] 100 % 0.0 b/s 0s 2.2MiB / 2.2MiB [========================================================================================================================] 100 % 0.0 b/s 0s 1.1MiB / 1.1MiB [========================================================================================================================] 100 % 0.0 b/s 0s 3.7MiB / 3.7MiB [========================================================================================================================] 100 % 0.0 b/s 0s 33.6MiB / 33.6MiB [======================================================================================================================] 100 % 0.0 b/s 0s 1.0MiB / 1.0MiB [========================================================================================================================] 100 % 0.0 b/s 0s 711.8KiB / 711.8KiB [====================================================================================================================] 100 % 0.0 b/s 0s 165.3MiB / 165.3MiB [====================================================================================================================] 100 % 0.0 b/s 0s 30.8MiB / 30.8MiB [======================================================================================================================] 100 % 0.0 b/s 0s 40.0MiB / 40.0MiB [======================================================================================================================] 100 % 0.0 b/s 0s 3.1MiB / 3.1MiB [========================================================================================================================] 100 % 0.0 b/s 0s 1007.5KiB / 1007.5KiB [==================================================================================================================] 100 % 0.0 b/s 0s 888.7KiB / 888.7KiB [====================================================================================================================] 100 % 0.0 b/s 0s 39.8MiB / 39.8MiB [======================================================================================================================] 100 % 0.0 b/s 0s 1.3MiB / 1.3MiB [========================================================================================================================] 100 % 0.0 b/s 0s 75.5KiB / 75.5KiB [======================================================================================================================] 100 % 0.0 b/s 0s 165.9MiB / 165.9MiB [====================================================================================================================] 100 % 0.0 b/s 0s 43.2MiB / 43.2MiB [======================================================================================================================] 100 % 0.0 b/s 0s 99.4MiB / 99.4MiB [======================================================================================================================] 100 % 0.0 b/s 0s 574.9KiB / 574.9KiB [====================================================================================================================] 100 % 0.0 b/s 0s 990.0KiB / 990.0KiB [====================================================================================================================] 100 % 0.0 b/s 0s 2.2MiB / 2.2MiB [========================================================================================================================] 100 % 0.0 b/s 0s 75.5KiB / 75.5KiB [======================================================================================================================] 100 % 0.0 b/s 0s 254.5MiB / 254.5MiB [====================================================================================================================] 100 % 0.0 b/s 0s 350.3MiB / 350.3MiB [====================================================================================================================] 100 % 0.0 b/s 0s INFO: Extracting OCI image... FATAL: While making image from oci registry: error fetching image to cache: while building SIF from layers: packer failed to pack: while unpacking tmpfs: error unpacking rootfs: unpack entry: opt/conda/pkgs/certifi-2018.11.29-py37_0/lib/python3.7/site-packages/certifi/__init__.py: link: unpriv.link: unpriv.wrap target: no such file or directory

Running in sudo, trying other versions of unify (1.0.8-1.1.2rc), clearing singularity cache, updating python and installing conda had no effect on the error. I should note that I am attempting to pull the image to an AWS instance. Pulling directly to docker on my local machine did not result in any issues, leading me to believe the issue lies in singularity attempting to build the .sif file. Any help here is appreciated.

[dtrudg]

Hmmm - this is a difficult one. The error is being reported from the dependency (opencontainers/umoci) that SingularityCE uses to extract the OCI container image.

It'll need careful examination of the image, and testing with umoci alone to work out what the issue is, and report it / hopefully provide a fix to the upstream project.

I'm afraid this isn't likely to be very quick... I'll write it up as an issue next week 🤞

[dtrudg]

I've created #2684 where I'll track the investigation of this issue.

As above, it's quite likely to be a problem in a dependency, so we'll have to work out a simple test case to report and discuss upstream.

[cfluta-latus]

Thanks so much for the quick response on this. Let me know if I can provide any more information to assist you!

[dtrudg]

I've found the problem. It's actually one step earlier than I thought at first... and easier to fix (for Singularity's native runtime, at least).

The fix will be in the next release, which will probably go out on Tuesday or Thursday next week.