[BUG] Faild to build the model from the docker file: Errors were encountered while processing: openjdk-11-jre-headless:amd64 #11801

ltrojan · 2024-04-23T15:36:20Z

Issues Policy acknowledgement

I have read and agree to submit bug reports in accordance with the issues policy

Where did you encounter this bug?

Local machine

Willingness to contribute

Yes. I can contribute a fix for this bug independently.

MLflow version

Client: N/A
Tracking server: 2.11.1

System information

system: N/A (Dockerfile: FROM ubuntu:20.04):
python: N/A (Dockerfile: : FROM ubuntu:20.04)

Describe the problem

the docker build . fails to run

steps

A tar file is created using the following command:

from mlflow.models.flavor_backend_registry import get_flavor_backend

backend = get_flavor_backend(
    tracker_uri,
    docker_build=True,
    env_manager=_env_manager)
backend.generate_dockerfile(
    tracker_uri,
    output_directory,
    mlflow_home=None,
    install_mlflow=False,
    enable_mlserver=False)

the commands generate a folder with the following files:

ltrojan@:tmp/$ ls -l
total 20
-rw-r--r-- 1 ltrojan ltrojan 2434 Apr 22 14:19 Dockerfile
drwxr-xr-x 3 ltrojan ltrojan 4096 Apr 22 14:19 model_dir/

and the Dockerfile:

ltrojan@:tmp/$ head Dockerfile

# Build an image that can serve mlflow models.
FROM ubuntu:20.04

RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends          wget          curl  
        nginx          ca-certificates          bzip2          build-essential          cmake          openjdk-8-jdk  
        git-core          maven     && rm -rf /var/lib/apt/lists/*


# Setup pyenv
RUN apt -y update

the build command is issued:

docker build .

this run for a min or two untill if fails with the following error:

94.93
95.24 done.
95.24 done.
95.26 Setting up openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~20.04) ...
95.30 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode
95.30 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java to provide /usr/bin/java (java) in auto mode
95.30 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode
95.30 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/jjs to provide /usr/bin/jjs (jjs) in auto mode
95.30 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/pack200 to provide /usr/bin/pack200 (pack200) in auto mode
95.31 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode
95.31 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/unpack200 to provide /usr/bin/unpack200 (unpack200) in auto mode
95.31 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode
95.31 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode
95.31 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode
95.62 Setting up openjdk-11-jre-headless:amd64 (11.0.22+7-0ubuntu2~20.04.1) ...
95.66 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode
95.66 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/jjs to provide /usr/bin/jjs (jjs) in auto mode
95.66 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode
95.67 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode
95.67 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode
95.67 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/pack200 to provide /usr/bin/pack200 (pack200) in auto mode
95.67 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/unpack200 to provide /usr/bin/unpack200 (unpack200) in auto mode
95.67 update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode
95.67 update-alternatives: error: error creating symbolic link '/usr/share/binfmts/jar.dpkg-tmp': No such file or directory
95.67 dpkg: error processing package openjdk-11-jre-headless:amd64 (--configure):
95.67  installed openjdk-11-jre-headless:amd64 package post-installation script subprocess returned error exit status 2
95.67 Setting up openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~20.04) ...
95.68 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode
95.68 Setting up openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~20.04) ...
95.69 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode
95.69 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode
95.69 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/rmic to provide /usr/bin/rmic (rmic) in auto mode
95.70 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode
95.70 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode
95.70 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode
95.71 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode
95.71 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javac to provide /usr/bin/javac (javac) in auto mode
95.71 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode
95.71 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode
95.71 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode
95.72 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode
95.72 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode
95.72 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode
95.72 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode
95.73 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode
95.73 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javap to provide /usr/bin/javap (javap) in auto mode
95.73 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jar to provide /usr/bin/jar (jar) in auto mode
95.73 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode
95.74 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode
95.74 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jps to provide /usr/bin/jps (jps) in auto mode
95.75 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode
95.75 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode
95.75 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode
95.75 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode
95.76 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode
95.76 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode
95.77 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jfr to provide /usr/bin/jfr (jfr) in auto mode
95.77 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode
95.77 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode
95.78 Setting up openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~20.04) ...
95.78 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode
95.79 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
95.79 Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.40.0+dfsg-3ubuntu0.4) ...
95.81 Processing triggers for libc-bin (2.31-0ubuntu9.14) ...
95.83 Errors were encountered while processing:
95.83  openjdk-11-jre-headless:amd64
95.86 E: Sub-process /usr/bin/dpkg returned an error code (1)
------
Dockerfile:6
--------------------
   4 |
   5 |     RUN apt-get -y update
   6 | >>> RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends          wget          curl          nginx          ca-certificates          bzip2          build-essential          cmake          openjdk-8-jdk          git-core          maven     && rm -rf /var/lib/apt/lists/*
   7 |
   8 |
--------------------
ERROR: failed to solve: process "/bin/sh -c DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends          wget          curl          nginx          ca-certificates          bzip2          build-essential          cmake          openjdk-8-jdk          git-core          maven     && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100

What component(s) does this bug affect?

What interface(s) does this bug affect?

area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
area/windows: Windows support

What language(s) does this bug affect?

language/r: R APIs and clients
language/java: Java APIs and clients
language/new: Proposals for new client languages

What integration(s) does this bug affect?

integrations/azure: Azure and Azure ML integrations
integrations/sagemaker: SageMaker integrations
integrations/databricks: Databricks integrations

The text was updated successfully, but these errors were encountered:

ltrojan · 2024-04-23T22:51:04Z

I did some additional investigation and it looks like the problem is resolved when installing openjdk-8-jdk separately: i replaced this:

RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake openjdk-8-jdk git-core maven && rm -rf /var/lib/apt/lists/*

with these two:

RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake git-core maven

RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends openjdk-8-jdk && rm -rf /var/lib/apt/lists/*

and that seems to fix the problem...

this however is not a feasible solution as editing the Dockerfile before being able to build the image seems very overkill...

daniellok-db · 2024-04-23T23:22:45Z

@ltrojan interesting, do you have any idea why it succeeds when you install openjdk-8-jdk after everything else? does it also succeed if you put openjdk-8-jdk at the end of the installs, like this?

RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake git-core maven openjdk-8-jdk && rm -rf /var/lib/apt/lists/

cc @harupy do you think the migration to the JDK 11 image has anything to do with it?

ltrojan · 2024-04-24T08:35:20Z

no, it's very weird...

however, i just realised those dockerfiles are created using a slightly old version of mlflow before PR #10954 where the following change was added to docker_utils.py:

RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends \
         wget \
         curl \
         nginx \
         ca-certificates \
         bzip2 \
         build-essential \
         cmake \
         openjdk-8-jdk \
         git-core \
         maven \
    && rm -rf /var/lib/apt/lists/*

# ...

for this further down the code (in generate_dockerfile())

        setup_java_steps = (
            "# Setup Java\n"
            "RUN apt-get install -y --no-install-recommends openjdk-8-jdk maven\n"
            "ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64"
        )

that looks like the packages openjdk-8-jdk and maven should be installed separately (there's a bunch of examples in the tests/resources/dockerfile folders in the same PR...

this is still spitting out the following:

root@c076165269b2:/# apt-get install -y --no-install-recommends openjdk-8-jdk maven

...

Processing triggers for libc-bin (2.31-0ubuntu9.14) ...
Setting up openjdk-11-jre-headless:amd64 (11.0.22+7-0ubuntu2~20.04.1) ...
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/jjs to provide /usr/bin/jjs (jjs) in auto mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/keytool to provide /usr/bin/keytool (keytool) in aut
o mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregi
stry) in auto mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/pack200 to provide /usr/bin/pack200 (pack200) in aut
o mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/unpack200 to provide /usr/bin/unpack200 (unpack200) 
in auto mode
update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode
update-alternatives: error: error creating symbolic link '/usr/share/binfmts/jar.dpkg-tmp': No such file or directory
dpkg: error processing package openjdk-11-jre-headless:amd64 (--configure):
 installed openjdk-11-jre-headless:amd64 package post-installation script subprocess returned error exit status 2
Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.40.0+dfsg-3ubuntu0.4) ...
Errors were encountered while processing:
 openjdk-11-jre-headless:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)

so, it seems the problem is related to installing openjdk-8-jdk, maven them together...

Beramos · 2024-04-29T09:48:59Z

Same issue here:

sysinfo:
Linux P14SGen2 5.14.0-1059-oem #67-Ubuntu SMP Mon Mar 13 14:22:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Beramos · 2024-04-29T10:03:44Z

I did some additional investigation and it looks like the problem is resolved when installing openjdk-8-jdk separately: i replaced this:
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake openjdk-8-jdk git-core maven && rm -rf /var/lib/apt/lists/*
with these two:
RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake git-core maven

RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends openjdk-8-jdk && rm -rf /var/lib/apt/lists/*
and that seems to fix the problem...

this however is not a feasible solution as editing the Dockerfile before being able to build the image seems very overkill...

This fix worked for me.

github-actions · 2024-05-01T00:14:03Z

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

ltrojan added the bug Something isn't working label Apr 23, 2024

github-actions bot added area/docker Docker use anywhere, such as MLprojects and MLmodels area/models MLmodel format, model serialization/deserialization, flavors labels Apr 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Faild to build the model from the docker file: Errors were encountered while processing: openjdk-11-jre-headless:amd64 #11801

[BUG] Faild to build the model from the docker file: Errors were encountered while processing: openjdk-11-jre-headless:amd64 #11801

ltrojan commented Apr 23, 2024 •

edited

ltrojan commented Apr 23, 2024 •

edited

daniellok-db commented Apr 23, 2024

ltrojan commented Apr 24, 2024

Beramos commented Apr 29, 2024

Beramos commented Apr 29, 2024

github-actions bot commented May 1, 2024

[BUG] Faild to build the model from the docker file: Errors were encountered while processing: openjdk-11-jre-headless:amd64 #11801

[BUG] Faild to build the model from the docker file: Errors were encountered while processing: openjdk-11-jre-headless:amd64 #11801

Comments

ltrojan commented Apr 23, 2024 • edited

Issues Policy acknowledgement

Where did you encounter this bug?

Willingness to contribute

MLflow version

System information

Describe the problem

steps

What component(s) does this bug affect?

What interface(s) does this bug affect?

What language(s) does this bug affect?

What integration(s) does this bug affect?

ltrojan commented Apr 23, 2024 • edited

daniellok-db commented Apr 23, 2024

ltrojan commented Apr 24, 2024

Beramos commented Apr 29, 2024

Beramos commented Apr 29, 2024

github-actions bot commented May 1, 2024

ltrojan commented Apr 23, 2024 •

edited

ltrojan commented Apr 23, 2024 •

edited