Static binary via pyoxidizer

[ramaro]

Explore building a kapitan binary using https://pyoxidizer.readthedocs.io/en/stable/index.html

[jkrzemin]

I've started fiddeling around, but so far I couldn't make it work as a static binary. During ciffi compilation it cannot find a lib

        musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -I/tools/deps/libedit/include -static -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -I/root/.cache/pyoxidizer/python_distributions/python.771eaf1ed737/python/install/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.10/c/_cffi_backend.o
        clang: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
        clang: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
        clang: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
        clang: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
        clang: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
        In file included from c/_cffi_backend.c:2:
        In file included from /root/.cache/pyoxidizer/python_distributions/python.771eaf1ed737/python/install/include/python3.10/Python.h:11:
        /usr/include/limits.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
        #include <bits/libc-header-start.h>
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
        1 error generated.
        error: command '/usr/bin/musl-clang' failed with exit code 1
        [end of output]

Thoughts after that that might save some time for the others:

• we will need to get rid of __file__ usage since pyoxidizer makes it dissapear
• for static compile it requires compiling against musl

If somebody want to take a shot at this ciffi issue, here is how to set yourself up.
Write the dockerfile for build image

FROM rust:latest
ENV DEBIAN_FRONTEND noninteractive
RUN apt update && apt install -y clang binutils --no-install-recommends && rm -rf /var/lib/apt/lists/*
RUN curl -LO http://www.musl-libc.org/releases/musl-1.2.2.tar.gz && tar -xvf musl-1.2.2.tar.gz
WORKDIR musl-1.2.2
RUN ./configure --prefix=/usr --enable-wrapper=all && make && make install
WORKDIR /opt/pyoxidizer
ADD https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.24.0/pyoxidizer-0.24.0-x86_64-unknown-linux-musl.tar.gz /opt/pyoxidizer/pyoxidizer-0.24.0-x86_64-unknown-linux-musl.tar.gz
RUN tar xzvf /opt/pyoxidizer/pyoxidizer-0.24.0-x86_64-unknown-linux-musl.tar.gz
RUN cp -v /opt/pyoxidizer/pyoxidizer-0.24.0-x86_64-unknown-linux-musl/pyoxidizer /usr/bin/pyoxidizer
RUN rustup target add x86_64-unknown-linux-musl

WORKDIR /src
COPY . .

CMD ["pyoxidizer", "build", "--release", "--target-triple", "x86_64-unknown-linux-musl"]

Write pyoxidizer.bzl for build configuration

def make_exe():
    dist = default_python_distribution(flavor="standalone_static")
    policy = dist.make_python_packaging_policy()
    policy.allow_files = True
    policy.include_non_distribution_sources = True
    policy.resources_location_fallback = "filesystem-relative:prefix"
    python_config = dist.make_python_interpreter_config()
    python_config.allocator_backend = "default"
    python_config.multiprocessing_start_method = 'auto'
    python_config.run_module = "kapitan"
    exe = dist.to_python_executable(
        name="kapitan",
        packaging_policy=policy,
        config=python_config,
    )
    exe.add_python_resources(exe.pip_install(["."]))
    return exe

def make_embedded_resources(exe):
    return exe.to_embedded_resources()

def make_install(exe):
    files = FileManifest()
    files.add_python_resource(".", exe)
    return files

register_target("exe", make_exe)
register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
register_target("install", make_install, depends=["exe"], default=True)
resolve_targets()

When you build the image it already has sources included and command set to compile, but if you want to play around live you can mount it and compile from hand.

Example:

➜  kapitan git:(master) ✗ docker build --no-cache -t builder -f Dockerfile.pyoxidize .
[+] Building 138.2s (18/18) FINISHED                                                                                                   
 => [internal] load build definition from Dockerfile.pyoxidize                                                                    0.1s
 => => transferring dockerfile: 48B                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                 0.0s
 => => transferring context: 2B                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/rust:latest                                                                    0.0s
 => CACHED https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.24.0/pyoxidizer-0.24.0-x86_64-unknown-linux-m  0.0s
 => [internal] load build context                                                                                                 0.3s
 => => transferring context: 329.99kB                                                                                             0.3s
 => CACHED [ 1/12] FROM docker.io/library/rust:latest                                                                             0.0s
 => [ 2/12] RUN apt update && apt install -y clang binutils --no-install-recommends && rm -rf /var/lib/apt/lists/*               37.4s
 => [ 3/12] RUN curl -LO http://www.musl-libc.org/releases/musl-1.2.2.tar.gz && tar -xvf musl-1.2.2.tar.gz                        9.5s
 => [ 4/12] WORKDIR musl-1.2.2                                                                                                    0.1s 
 => [ 5/12] RUN ./configure --prefix=/usr --enable-wrapper=all && make && make install                                           58.3s 
 => [ 6/12] WORKDIR /opt/pyoxidizer                                                                                               0.1s 
 => [ 7/12] ADD https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.24.0/pyoxidizer-0.24.0-x86_64-unknown-li  0.2s 
 => [ 8/12] RUN tar xzvf /opt/pyoxidizer/pyoxidizer-0.24.0-x86_64-unknown-linux-musl.tar.gz                                       1.4s 
 => [ 9/12] RUN cp -v /opt/pyoxidizer/pyoxidizer-0.24.0-x86_64-unknown-linux-musl/pyoxidizer /usr/bin/pyoxidizer                  0.8s 
 => [10/12] RUN rustup target add x86_64-unknown-linux-musl                                                                      24.0s 
 => [11/12] WORKDIR /src                                                                                                          0.1s 
 => [12/12] COPY . .                                                                                                              3.3s 
 => exporting to image                                                                                                            2.6s
 => => exporting layers                                                                                                           2.5s
 => => writing image sha256:a2f21c5b32a183f6dcb7e61f4591f0f6136e7236d6f720eb16c6914bede7a00f                                      0.0s
 => => naming to docker.io/library/builder                                                                                        0.0s
(kapitan) ➜  kapitan git:(master) ✗ docker run -it -v $(pwd):/src builder bash                  
root@21b946afc897:/src# pyoxidizer build --release --target-triple x86_64-unknown-linux-musl

[jkrzemin]

I've created an issue in pyoxidizer repository - maybe somebody will help us with this one (I'm out of ideas).

indygreg/PyOxidizer#685

[gruzewski]

@jkrzemin thanks for the investigation. I tried to pick it up from where you have finished with some progress.

The __file__ issue can be avoided if we add files alongside the binary with register_resource_callback. All the Kapitan's usage of __file__ should be replaced following this guideline.

Unfortunately, I had issues adding reclass to the binary. At the moment it fails with:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "kapitan.cli", line 23, in <module>
  File "kapitan.inputs.jsonnet", line 15, in <module>
  File "kapitan.resources", line 28, in <module>
  File "reclass", line 14, in <module>
  File "reclass.cli", line 16, in <module>
ImportError: cannot import name 'get_storage' from partially initialized module 'reclass' (most likely due to a circular import) (/Users/jacek/Code/kapitan_fork/build/aarch64-apple-darwin/debug/install/lib/reclass/__init__.py)

In regards to using Docker for compilation, I am not sure if cross-platform compilation would work on MacOS (more info in PyOxidizer/issues/566). We would need to use different workers for that.

My attempt is here if everyone wants to give it a go.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.