Skip to content

Commit

Permalink
docs: Buildah articles & run in container (#4043)
Browse files Browse the repository at this point in the history
Correcting & translating Buildah and Run in Kubernetes articles
  • Loading branch information
kirkonru committed Dec 23, 2021
1 parent b66cca5 commit 32cad28
Show file tree
Hide file tree
Showing 14 changed files with 926 additions and 136 deletions.
30 changes: 15 additions & 15 deletions docs/pages_en/advanced/buildah_mode.md
Expand Up @@ -3,32 +3,32 @@ title: Buildah mode
permalink: advanced/buildah_mode.html
---

werf currently supports building of images _with docker server_ or _without docker server_ (in experimental mode). This page contains information, which is only applicable for experimental mode _without docker server_. Only dockerfile-images builder is available for this mode for now. Stapel-images builder will be available soon.
werf currently supports building images _with the Docker server_ or _without the Docker server_ (in experimental mode). This page contains information applicable only to the experimental mode _without the Docker server_. For now, only the Dockerfile image builder is available for this mode. The Stapel image builder will be available soon.

In experimental mode _without docker server_ werf uses built-in buildah in rootless mode.
In the experimental mode _without the Docker server_, werf uses built-in Buildah in rootless mode.

## Enable buildah
## Enable Buildah

Buildah is enabled by setting `WERF_BUILDAH_MODE` environment variable to one of the options: `auto`, `native-chroot`, `native-rootless` or `docker-with-fuse`.
Buildah is enabled by setting the `WERF_BUILDAH_MODE` environment variable to one of the following: `auto`, `native-chroot`, `native-rootless` or `docker-with-fuse`.

* `auto` — select mode automatically based on your platform and environment.
* `native-chroot` works only in Linux and uses `chroot` isolation level when running build containers.
* `native-rootless` works only in Linux and uses `rootless` isolation leven when running build containers. In this isolation level werf will use container runtime (runc or crun).
* `docker-with-fuse` is crossplatform mode, and it is the only available choice in MacOS or Windows.
* `auto` — select the mode automatically based on your platform and environment.
* `native-chroot` works only on Linux and uses the `chroot` isolation level when running build containers.
* `native-rootless` works only on Linux and uses the `rootless` isolation level when running build containers. At this isolation level werf will use container runtime (runc or crun).
* `docker-with-fuse` is a cross-platform mode and is the only choice available on MacOS or Windows.

Most users should just set `WERF_BUILDAH_MODE=auto` to enable experimental buildah mode.
Most users only need to set `WERF_BUILDAH_MODE=auto` to enable the experimental Buildah-based mode.

## Storage driver

Werf may use `overlay` or `vfs` storage driver:
werf can use `overlay` or `vfs` storage driver:

* `overlay` enables usage of overlayfs filesystem. Either native linux kernel overlayfs could be used if available or fuse-overlayfs. It is the default and recommended choice.
* `vfs` enables usage of virtual filesystem emulation instead of overlayfs. This filesystem has worse performance and requires privileged container, thus it is not recommended. But it may be required in some cases.
* `overlay` allows you to use the OverlayFS filesystem. You can either use the native Linux kernel's OverlayFS (if available) or fuse-overlayfs. It is the default and recommended choice.
* `vfs` allows you to use a virtual filesystem emulation instead of OverlayFS. This filesystem has worse performance and requires a privileged container, so its use is not recommended. However, it may be required in some cases.

Generally user should just use default `overlay`. Storage driver could be selected with the `WERF_BUILDAH_STORAGE_DRIVER` environment variable.
Normally, the user should just go with the default `overlay` driver. The storage driver can be selected with the `WERF_BUILDAH_STORAGE_DRIVER` environment variable.

### System requirements

Starting with Linux kernel version 5.11 rootless overlayfs is available (technically from 5.13 version, which contains bugfix to enable rootless overlayfs with SELinux, but most major linux distributions has been backported this bugfix to 5.11 kernel).
Rootless OverlayFS is available starting with Linux kernel version 5.11 (strictly speaking, it was initially implemented in version 5.13 via a bugfix to enable rootless overlayFS in SELinux, but most major Linux distributions have backported it into kernel 5.11).

If your kernel does not support rootless overlayfs, then fuse-overlayfs will be used.
If your kernel does not support rootless OverlayFS, fuse-overlayfs will be used.
46 changes: 23 additions & 23 deletions docs/pages_en/advanced/ci_cd/run_in_container/how_it_works.md
Expand Up @@ -3,40 +3,40 @@ title: How it works
permalink: advanced/ci_cd/run_in_container/how_it_works.html
---

werf currently supports building of images _with docker server_ or _without docker server_ (in experimental mode). This page contains information, which is only applicable for experimental mode _without docker server_. Only dockerfile-images builder is available for this mode for now. Stapel-images builder will be available soon.
werf currently supports building images _with the Docker server_ or _without the Docker server_ (in experimental mode). This page contains information applicable only to the experimental mode _without the Docker server_. For now, only the Dockerfile image builder is available for this mode. The Stapel image builder will be available soon.

General info how to enable buildah in werf is available at [buildah mode page]({{ "/advanced/buildah_mode.html" | true_relative_url }}).
General info on how to enable Buildah in werf is available on the [Buildah mode page]({{ "/advanced/buildah_mode.html" | true_relative_url }}).

## Modes of operation

Depending on user system conformance with [buildah mode system requirements]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and depending on user needs there are 3 ways to use werf with buildah inside containers:
Depending on whether the user's system meets the [Buildah mode system requirements]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and depending on the user needs, there are 3 ways to use werf with Buildah inside containers:

1. Use [Linux kernel with rootless overlayfs]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}).
2. Use [Linux kernel without rootless overlayfs]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and privileged container.
3. Use [Linux kernel without rootless overlayfs]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and non-privileged container with additional settings.
1. Use [Linux kernel with rootless OverlayFS]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}).
2. Use [Linux kernel without rootless OverlayFS]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and a privileged container.
3. Use [Linux kernel without rootless OverlayFS]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}) and an unprivileged container with additional settings.

### Linux kernel with rootless overlayfs
### Linux kernel with rootless OverlayFS

This way you only required to **disable apparmor** and **seccomp** profiles in a container which runs werf.
In this case, you only need to **disable the AppArmor** and **seccomp** profiles in the container where werf is running.

### Linux kernel without rootless overlayfs and privileged container
### Linux kernel without rootless OverlayFS and privileged container

In the case your Linux kernel does not have [rootless overlayfs]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}), buildah and werf uses fuse-overlayfs instead.
If your Linux kernel does not have [rootless OverlayFS]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}), Buildah and werf will use fuse-overlayfs instead.

To enable usage of fuse-overlayfs in such case using a **privileged container** to run werf is one option.
One of the options in this case is to run werf in a **privileged container** to enable fuse-overlayfs.

### Linux kernel without rootless overlayfs and non-privileged container
### Linux kernel without rootless OverlayFS and non-privileged container

In the case your Linux kernel does not have [rootless overlayfs]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}), buildah and werf uses fuse-overlayfs instead.
If your Linux kernel does not support [OverlayFS in the rootless mode]({{ "/advanced/buildah_mode.html#system-requirements" | true_relative_url }}), Buildah and werf will use fuse-overlayfs instead.

To enable usage of fuse-overlayfs in such case without using a privileged container, run werf in a container with the following options:
To use fuse-overlayfs without a privileged container, run werf in a container with the following parameters:

1. **Disabled apparmor** and **seccomp** profiles .
2. Enabled `/dev/fuse` device.
1. The **AppArmor** and **seccomp** profiles are **disabled**.
2. The `/dev/fuse` device is enabled.

## Available werf images

Following images are provided with builtin werf. Each image will be updated through the same release process as used in the trdl package manager, [more info about release channels]({{ "installation.html#all-changes-in-werf-go-through-all-release-channels" | true_relative_url }}).
Below is a list of images with built-in werf. Each image is updated as part of a release process based on the trdl package manager ([learn more about the release channels]({{ "installation.html#all-changes-in-werf-go-through-all-release-channels" | true_relative_url }}).

* `ghcr.io/werf/werf:latest` -> `ghcr.io/werf/werf:1.2-stable`;
* `ghcr.io/werf/werf:1.2-alpha` -> `ghcr.io/werf/werf:1.2-alpha-alpine`;
Expand Down Expand Up @@ -64,7 +64,7 @@ Following images are provided with builtin werf. Each image will be updated thro

### `fuse: device not found`

Full error may look like:
The complete error message may look as follows:

```
error mounting new container: error mounting build container "53f916e7a334a4bb0d9dbc38a0901718d40b99765002bb7f2f2e5464b1db4294": error creating overlay mount to /home/build/.local/share/containers/storage/overlay/49e856f537ba58afdc09137291133994cd1305e40df72c4fab43077cbd405477/merged, mount_data=",lowerdir=/home/build/.local/share/containers/storage/overlay/l/Z5GEVIFIIQ7H262DYUTX3YOVR6:/home/build/.local/share/containers/storage/overlay/l/PJBBW6UNUNGI37IX6R3LDNPX3J:/home/build/.local/share/containers/storage/overlay/l/MUYSUONLQVE4CJMQVDCH2UBAVQ:/home/build/.local/share/containers/storage/overlay/l/67JHKJDCKBTI4R3Q5S5YG44AD3:/home/build/.local/share/containers/storage/overlay/l/3S72G4SWKDXILGANUOCESP5LDK,upperdir=/home/build/.local/share/containers/storage/overlay/49e856f537ba58afdc09137291133994cd1305e40df72c4fab43077cbd405477/diff,workdir=/home/build/.local/share/containers/storage/overlay/49e856f537ba58afdc09137291133994cd1305e40df72c4fab43077cbd405477/work,volatile": using mount program /usr/bin/fuse-overlayfs: fuse: device not found, try 'modprobe fuse' first
Expand All @@ -73,11 +73,11 @@ fuse-overlayfs: cannot mount: No such file or directory
time="2021-12-06T11:30:20Z" level=error msg="exit status 1"
```

Solution: enable fuse device for a container which runs werf, [more info](#linux-kernel-without-rootless-overlayfs-and-non-privileged-container).
Solution: enable the fuse device for the container in which werf is running ([details](#linux-kernel-without-rootless-overlayfs-and-non-privileged-container)).

### `flags: 0x1000: permission denied`

Full error may look like:
The complete error message may look as follows:

```
time="2021-12-06T11:23:23Z" level=debug msg="unable to create kernel-style whiteout: operation not permitted"
Expand All @@ -88,16 +88,16 @@ Error: unable to get buildah client: unable to create new Buildah instance with
time="2021-12-06T11:23:23Z" level=error msg="exit status 1"
```

Solution: disable apparmor and seccomp profiles with options like `--security-opt seccomp=unconfined` and `--security-opt apparmor=unconfined` or by setting special annotations in a Pod, [more info](#linux-kernel-without-rootless-overlayfs-and-non-privileged-container)
Solution: disable AppArmor and seccomp profiles with `--security-opt seccomp=unconfined` and `--security-opt apparmor=unconfined` or add special annotations to a Pod ([details](#linux-kernel-without-rootless-overlayfs-and-non-privileged-container)).

### `unshare(CLONE_NEWUSER): Operation not permitted`

Full error may look like:
The complete error message may look as follows:

```
Error during unshare(CLONE_NEWUSER): Operation not permitted
ERRO[0000] error parsing PID "": strconv.Atoi: parsing "": invalid syntax
ERRO[0000] (unable to determine exit status)
```

Solution: disable apparmor and seccomp profiles with options like `--security-opt seccomp=unconfined` and `--security-opt apparmor=unconfined` or by setting special annotations in a Pod, or use a privileged container, [more info](#modes-of-operation).
Solution: disable AppArmor and seccomp profiles with `--security-opt seccomp=unconfined` and `--security-opt apparmor=unconfined`, add special annotations to a Pod or use a privileged container ([details](#modes-of-operation)).
@@ -1,41 +1,41 @@
---
title: Use docker container
title: Use Docker container
permalink: advanced/ci_cd/run_in_container/use_docker_container.html
---

> NOTICE: werf currently supports building of images with connection to the docker-server or without connection docker-server (in experimental mode). Building of images without docker-server is available in experimental mode for now and it is the only recommended way.
> NOTICE: werf currently supports building images _with the Docker server_ or _without the Docker server_ (in experimental mode). Building images without the Docker server is still experimental, however, it is the only recommended mode.
## Build images without docker server (NEW!)
## Build images without the Docker server (NEW!)

> NOTICE: Only dockerfile-images builder is available for this type of build for now. Stapel-images builder will be available soon.
> NOTICE: For now, only the Dockerfile image builder is available for this type of builds. The Stapel image builder will be available soon.
There is official image with werf 1.2 for this method (1.1 is not supported): `ghcr.io/werf/werf`.
There is an official image with werf 1.2 for this method (1.1 is not supported): `ghcr.io/werf/werf`.

Select and proceed with one of the [available modes of operation]({{ "advanced/ci_cd/run_in_container/how_it_works.html#modes-of-operation" | true_relative_url }}).
Select one of the [available operating modes]({{ "advanced/ci_cd/run_in_container/how_it_works.html#modes-of-operation" | true_relative_url }}).

### Linux kernel with rootless overlayfs
### Linux kernel with rootless OverlayFS

Only disable seccomp and apparmor profiles in such case. Example command:
In this case, you only need to disable the seccomp and AppArmor profiles. Below is an example of a command that does this:

```shell
docker run \
--security-opt seccomp=unconfined --security-opt apparmor=unconfined \
ghcr.io/werf/werf:latest WERF_COMMAND
```

### Linux kernel without rootless overlayfs and privileged container
### Linux kernel without rootless OverlayFS and privileged container

Just use privileged container in such case. Example command:
In this case, just use the privileged container. Below is an example of a command that does this:

```shell
docker run \
--privileged \
ghcr.io/werf/werf:latest WERF_COMMAND
```

### Linux kernel without rootless overlayfs and non-privileged container
### Linux kernel without rootless OverlayFS and non-privileged container

Disable seccomp and apparmor profiles and enable `/dev/fuse` in the container (so that `fuse-overlayfs` could work) in such case. Example command:
In this case, disable the seccomp and AppArmor profiles and enable `/dev/fuse` in the container (so that `fuse-overlayfs` can work). Below is an example of a command that does this:

```shell
docker run \
Expand All @@ -44,13 +44,13 @@ docker run \
ghcr.io/werf/werf:latest WERF_COMMAND
```

## Build images with docker server (not recommended)
## Build images with the Docker server (not recommended)

### Local docker server
### Local Docker server

This method supports building of dockerfile-images or stapel-images.
This method supports building Dockerfile images or Stapel images.

Example command:
Below is an example of a command that does this:

```shell
docker run \
Expand All @@ -61,17 +61,17 @@ docker run \
IMAGE WERF_COMMAND
```

Build your own docker image with werf for this method.
For this method, build your own Docker image using werf.

### Remote docker server
### Remote Docker server

This method supports only building of dockerfile-images. Stapel-images is not supported, because stapel images builder use mounts from host system into docker images.
This method only supports building Dockerfile images. Stapel images are not supported because the Stapel image builder uses mounts from the host system to Docker images.

The easiest way to use remote docker server inside docker container is docker-in-docker (dind).
The easiest way to use a remote Docker server inside a Docker container is Docker-in-Docker (dind).

Build your own docker image based on `docker:dind` for this method.
For this method, build your own image based on `docker:dind`.

Example command:
Below is an example of a command:

```shell
docker run \
Expand All @@ -83,4 +83,4 @@ Build your own docker image with werf for this method.

## Troubleshooting

If you have any problems please refer to the [troubleshooting section]({{ "advanced/ci_cd/run_in_container/how_it_works.html#troubleshooting" | true_relative_url }})
In case of problems, refer to the [troubleshooting section]({{ "advanced/ci_cd/run_in_container/how_it_works.html#troubleshooting" | true_relative_url }})

0 comments on commit 32cad28

Please sign in to comment.