[dind] Cannot get cgroups v2 working with rootless container #42910

dg424 · 2021-10-05T19:43:56Z

Hi,

Followed instructions here to setup host - https://docs.docker.com/engine/security/rootless/#limiting-resources

$ uname -a
Linux ubuntu 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

root@ubuntu:~# cat /sys/fs/cgroup/cgroup.controllers
cpuset cpu io memory hugetlb pids rdma

Here is the host docker info:

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 **Cgroup Driver: systemd
 Cgroup Version: 2**
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.11.0-37-generic
 Operating System: Ubuntu 20.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.64GiB
 Name: ubuntu
 ID: 7IXJ:75XQ:7U7X:QNM7:RSAE:VE3X:IWOU:OWEX:GHUR:MF2E:4YEZ:GXE2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Rootless info

$ docker run --rm -it --privileged --name dind-rootless -e DOCKER_TLS_CERTDIR="" -p 2375:2375 docker:dind-rootless --tls=false

$ export DOCKER_HOST=127.0.0.1:2375
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
  userxattr: true
 Logging Driver: json-file
 **Cgroup Driver: none**
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  rootless
  cgroupns
 Kernel Version: 5.11.0-37-generic
 Operating System: Alpine Linux v3.14
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.64GiB
 Name: 3413947f33b5
 ID: QDAZ:HRE3:2E4A:2N6R:LOVA:FSQ3:DCM3:ZXPH:G3QN:7OUG:7QEM:IQU5
 Docker Root Dir: /home/rootless/.local/share/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
**WARNING: Running in rootless-mode without cgroups. Systemd is required to enable cgroups in rootless-mode.**

The text was updated successfully, but these errors were encountered:

AkihiroSuda · 2021-10-06T12:37:53Z

Rootless cgroup v2 requires systemd, which is missing in the official dind, so you can’t enable rootless cgroup v2 with the official dind image.

AkihiroSuda · 2021-10-06T12:41:01Z

cc @tianon

Would it make sense to create the systemd variant of the dind image like docker:20.10-dind-rootless-systemd-bullseye ?

ddag · 2021-10-06T12:54:16Z

Ah, so this link - https://docs.docker.com/engine/security/rootless/#limiting-resources - is only for native non-docker install of rootless. Yes, a Debian image would be great for those of us running the rootless daemon as a docker container.

tianon · 2021-10-06T17:11:24Z

Having a Docker container with running/functioning systemd is definitely going to be a challenge, in my experience. 😕

dg424 · 2021-10-06T17:14:31Z

Is it a matter of time to do it or it's probably not possible technically ? Just wondering, if it's just not possible, then at least we can move ahead with this feature not being available for use by containers running in a rootless environment.

tianon · 2021-10-06T17:35:10Z

It's possible, but it's definitely going to be complicated and likely going to be really fragile.

dg424 · 2021-10-06T17:52:37Z

Ah, @tianon I know you can do it! 😃

dg424 · 2021-10-06T18:13:47Z

Wasn't there at one point a Debian build of the Docker container before it was moved to Alpine ?

tianon · 2021-10-06T18:37:35Z

To be perfectly clear, I have no interest in, nor plans to work on (or maintain) such a thing. 😬

Re: Debian, see docker-library/docker#306

dg424 · 2021-10-06T18:39:31Z

Ok. I'll take a look at the repo and see if I can somehow do something. Unless anyone else in the community has done the work already and I'm lucky enough that they see this post and help. Thanks.

dg424 · 2021-10-06T19:28:16Z

Hi @tianon, using your link here - docker-library/docker#306 (comment), I created an image based on rootless. I don't see - WARNING: Running in rootless-mode without cgroups. Systemd is required to enable cgroups in rootless-mode. A run of progrium/stress along with docker stats seems to verify resource limits are working ?

$ cat Dockerfile 
FROM debian:bullseye-slim

RUN set -eux; \
	apt-get update; \
	apt-get install -y --no-install-recommends \
		ca-certificates \
		iptables \
		openssl \
		pigz \
		xz-utils \
	; \
	rm -rf /var/lib/apt/lists/*

ENV DOCKER_TLS_CERTDIR=/certs
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client

COPY --from=docker:20.10.9-dind-rootless /usr/local/bin/ /usr/local/bin/

VOLUME /var/lib/docker

ENTRYPOINT ["dockerd-entrypoint.sh"]
CMD []

$ docker build -t dind-rootless-debian .

$ docker run --rm -d --privileged --name dind-rootless -e DOCKER_TLS_CERTDIR="" -p 2375:2375 dind-rootless-debian --tls=false

$ export DOCKER_HOST=127.0.0.1:2375
$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.11.0-37-generic
 Operating System: Debian GNU/Linux 11 (bullseye) (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.64GiB
 Name: dfaeecbb0ecc
 ID: 7S6J:OOKS:LI4E:Z4RM:2XNT:A3IU:65WM:XCG4:S6SO:3AMP:WVXK:FP4B
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/

$ docker run --rm -d -m 128m --cpus 0.5 progrium/stress -c 1 --vm 1 --vm-bytes 128m -t 30s

$ docker stats
CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT   MEM %     NET I/O   BLOCK I/O       PIDS
6e98473989b2   inspiring_khayyam   52.46%    127.9MiB / 128MiB   99.95%    0B / 0B   510MB / 740MB   3

tianon · 2021-10-06T21:08:21Z

That's not running rootless -- you'll need to adapt/add more of the bits from https://github.com/docker-library/docker/blob/8b8d62e7eb791b060cc75cb2956724a1bdc5484b/20.10/dind-rootless/Dockerfile to get it running in rootless mode as well.

dg424 · 2021-10-06T21:12:15Z

Ah, thanks. Let me get that in as well.

dg424 · 2021-10-06T21:49:57Z

Was able to get an image built, but this happens when I run rootless:

[rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 42 [0 1000 1 1 100000 65536 65537 100000 65536] failed: newuidmap: write to uid_map failed: Invalid argument

Maybe @AkihiroSuda knows about this one...

Dockerfile:

FROM debian:bullseye-slim

RUN set -eux; \
	apt-get update; \
	apt-get install -y --no-install-recommends \
	ca-certificates \
	iptables \
	openssl \
	pigz \
	xz-utils \
        iproute2 \
        uidmap \
        wget \
        xfsprogs \
	btrfs-progs \
	e2fsprogs \
	; \
	rm -rf /var/lib/apt/lists/*

ENV DOCKER_TLS_CERTDIR=/certs
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client

# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -eux; \
	addgroup --system dockremap; \
	adduser --system --ingroup dockremap dockremap; \
	echo 'dockremap:165536:65536' >> /etc/subuid; \
	echo 'dockremap:165536:65536' >> /etc/subgid

ENV DIND_COMMIT 42b1175eda071c0e9121e1d64345928384a93df1

RUN set -eux; \
	wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
	chmod +x /usr/local/bin/dind

COPY --from=docker:dind-rootless /usr/local/bin/ /usr/local/bin/

# "/run/user/UID" will be used by default as the value of XDG_RUNTIME_DIR
RUN mkdir /run/user && chmod 1777 /run/user

# create a default user preconfigured for running rootless dockerd
RUN set -eux; \
	useradd -m -c 'Rootless' -u 1000 rootless; \
	passwd -d rootless; \
	echo 'rootless:100000:65536' >> /etc/subuid; \
	echo 'rootless:100000:65536' >> /etc/subgid

RUN set -eux; \
	arch="$(uname -m)"; \
	case "$arch" in \
		'x86_64') \
			url='https://download.docker.com/linux/static/stable/x86_64/docker-rootless-extras-20.10.9.tgz'; \
			;; \
		'aarch64') \
			url='https://download.docker.com/linux/static/stable/aarch64/docker-rootless-extras-20.10.9.tgz'; \
			;; \
		*) echo >&2 "error: unsupported architecture ($arch)"; exit 1 ;; \
	esac; \
	\
	wget -O rootless.tgz "$url"; \
	\
	tar --extract \
		--file rootless.tgz \
		--strip-components 1 \
		--directory /usr/local/bin/ \
		'docker-rootless-extras/rootlesskit' \
		'docker-rootless-extras/rootlesskit-docker-proxy' \
		'docker-rootless-extras/vpnkit' \
	; \
	rm rootless.tgz; \
	\
	rootlesskit --version; \
	vpnkit --version

# pre-create "/var/lib/docker" for our rootless user
RUN set -eux; \
	mkdir -p /home/rootless/.local/share/docker; \
	chown -R rootless:rootless /home/rootless/.local/share/docker

VOLUME /home/rootless/.local/share/docker

VOLUME /var/lib/docker

EXPOSE 2375 2376

ENTRYPOINT ["dockerd-entrypoint.sh"]
CMD []

USER rootless

kernel unprivileged clone:

$ sysctl kernel.unprivileged_userns_clone
kernel.unprivileged_userns_clone = 1

rootlesskit installed with the Docker package:

rootless@fa28e6349e2e:/$ rootlesskit --version
rootlesskit version 0.14.2

dg424 · 2021-10-18T15:18:54Z

Update, see here - rootless-containers/rootlesskit#271 (comment)

Updated workfing Dockerfile for Debian Bullseye:

FROM debian:bullseye-slim

RUN set -eux; \
	apt-get update; \
	apt-get install -y --no-install-recommends \
	ca-certificates \
	iptables \
	openssl \
	pigz \
	xz-utils \
        iproute2 \
        uidmap \
        wget \
        xfsprogs \
	btrfs-progs \
	e2fsprogs \
	vim \
	; \
	rm -rf /var/lib/apt/lists/*

ENV DOCKER_TLS_CERTDIR=/certs
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client

# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -eux; \
	addgroup --system dockremap; \
	adduser --system --ingroup dockremap dockremap; \
	echo 'dockremap:165536:65536' >> /etc/subuid; \
	echo 'dockremap:165536:65536' >> /etc/subgid

ENV DIND_COMMIT 42b1175eda071c0e9121e1d64345928384a93df1

RUN set -eux; \
	wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
	chmod +x /usr/local/bin/dind

COPY --from=docker:dind-rootless /usr/local/bin/ /usr/local/bin/

# "/run/user/UID" will be used by default as the value of XDG_RUNTIME_DIR
RUN mkdir /run/user && chmod 1777 /run/user

# create a default user preconfigured for running rootless dockerd
RUN set -eux; \
	useradd -m -c 'Rootless' -u 1000 rootless; \
	passwd -d rootless;
#	echo 'rootless:100000:65536' >> /etc/subuid; \
#	echo 'rootless:100000:65536' >> /etc/subgid

RUN set -eux; \
	arch="$(uname -m)"; \
	case "$arch" in \
		'x86_64') \
			url='https://download.docker.com/linux/static/stable/x86_64/docker-rootless-extras-20.10.9.tgz'; \
			;; \
		'aarch64') \
			url='https://download.docker.com/linux/static/stable/aarch64/docker-rootless-extras-20.10.9.tgz'; \
			;; \
		*) echo >&2 "error: unsupported architecture ($arch)"; exit 1 ;; \
	esac; \
	\
	wget -O rootless.tgz "$url"; \
	\
	tar --extract \
		--file rootless.tgz \
		--strip-components 1 \
		--directory /usr/local/bin/ \
		'docker-rootless-extras/rootlesskit' \
		'docker-rootless-extras/rootlesskit-docker-proxy' \
		'docker-rootless-extras/vpnkit' \
	; \
	rm rootless.tgz; \
	rootlesskit --version; \
	vpnkit --version

# pre-create "/var/lib/docker" for our rootless user
RUN set -eux; \
	mkdir -p /home/rootless/.local/share/docker; \
	chown -R rootless:rootless /home/rootless/.local/share/docker

VOLUME /home/rootless/.local/share/docker

VOLUME /var/lib/docker

EXPOSE 2375 2376

ENTRYPOINT ["dockerd-entrypoint.sh"]
CMD []

USER rootless

thaJeztah · 2021-10-18T15:56:37Z

If you're building a debian-based image, might as well install the regular (non-static) .deb packages for Debian https://docs.docker.com/engine/install/debian/

dg424 · 2021-10-18T15:57:08Z

Ok, even with a successful build and run, docker stats reports incorrect data which I assume means that resource limiting is still not working.

Build image using Dockerfile from above:
# docker build -t dind-rootless-debian .

terminal1:
# docker run --rm -it --privileged --name dind-rootless -e TINI_SUBREAPER=1 -e DOCKER_TLS_CERTDIR="" -p 2375:2375 dind-rootless-debian /usr/local/bin/dockerd-entrypoint.sh --tls=false

terminal2:
# export DOCKER_HOST=127.0.0.1:2375
# docker run --rm -it -m 12m ubuntu bash
root@f10dda5b83fd:/#

terminal3:
# docker stats --no-stream
CONTAINER ID   NAME               CPU %     MEM USAGE / LIMIT     MEM %     NET I/O       BLOCK I/O       PIDS
3a28cc090330   sleepy_chaplygin   0.78%     117.2MiB / 15.64GiB   0.73%     1.25kB / 0B   844kB / 108MB   62

Now, with  the host docker daemon, same thing, works:

# docker run --rm -it -m 12m ubuntu bash
root@3113d8d366fd:/#

# docker stats --no-stream
CONTAINER ID   NAME               CPU %     MEM USAGE / LIMIT   MEM %     NET I/O       BLOCK I/O   PIDS
3113d8d366fd   quirky_blackwell   0.00%     804KiB / 12MiB      6.54%     2.96kB / 0B   0B / 0B     1

dg424 · 2021-10-18T15:58:27Z

Hi Sebastian,

Looking at the official Dockerfiles, the only thing that I see that is not from the distro's repo is this:

https://download.docker.com/linux/static/stable/x86_64/docker-rootless-extras-20.10.9.tgz

and this:

ENV DIND_COMMIT 42b1175eda071c0e9121e1d64345928384a93df1

RUN set -eux; \
	wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
	chmod +x /usr/local/bin/dind

Is there a deb package for the above ?

thaJeztah · 2021-10-18T16:12:14Z

The official Docker images install the static binaries; https://github.com/docker-library/docker/blob/8b8d62e7eb791b060cc75cb2956724a1bdc5484b/20.10/dind-rootless/Dockerfile#L25-L32

Is there a deb package for the above ?

No, there's no deb package for the dind script; it's just a shell script (https://github.com/moby/moby/blob/v20.10.9/hack/dind)

dg424 · 2021-10-18T17:55:31Z

Ok, here is the working Dockerfile using packages from the Debian repo:

FROM debian:bullseye-slim

RUN apt-get update && apt-get install -y \
	apt-transport-https \
	ca-certificates \
	curl \
	gnupg \
	lsb-release \
	slirp4netns \
	iptables \
	openssl \
	pigz \
	xz-utils \
        iproute2 \
        uidmap \
        wget \
        xfsprogs \
	btrfs-progs \
	e2fsprogs \
	vim \
	; \
	rm -rf /var/lib/apt/lists/*

RUN curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
	echo \
		"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
		$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
	apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io docker-ce-rootless-extras && \
	wget -O /usr/local/bin/docker-entrypoint.sh https://raw.githubusercontent.com/docker-library/docker/master/20.10/docker-entrypoint.sh && \
	wget -O /usr/local/bin/modprobe.sh https://raw.githubusercontent.com/docker-library/docker/master/20.10/modprobe.sh && \
	wget -O /usr/local/bin/dockerd-entrypoint.sh https://raw.githubusercontent.com/docker-library/docker/master/20.10/dind/dockerd-entrypoint.sh && \
	chmod +x /usr/local/bin/*

ENV DOCKER_TLS_CERTDIR=/certs
RUN mkdir /certs /certs/client && chmod 1777 /certs /certs/client

# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -eux; \
	addgroup --system dockremap; \
	adduser --system --ingroup dockremap dockremap; \
	echo 'dockremap:165536:65536' >> /etc/subuid; \
	echo 'dockremap:165536:65536' >> /etc/subgid

ENV DIND_COMMIT 42b1175eda071c0e9121e1d64345928384a93df1

RUN set -eux; \
	wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
	chmod +x /usr/local/bin/dind

# "/run/user/UID" will be used by default as the value of XDG_RUNTIME_DIR
RUN mkdir /run/user && chmod 1777 /run/user

# create a default user preconfigured for running rootless dockerd
RUN set -eux; \
	useradd -m -c 'Rootless' -u 1000 rootless; \
	passwd -d rootless;

# pre-create "/var/lib/docker" for our rootless user
RUN set -eux; \
	mkdir -p /home/rootless/.local/share/docker; \
	chown -R rootless:rootless /home/rootless/.local/share/docker

VOLUME /home/rootless/.local/share/docker

VOLUME /var/lib/docker

EXPOSE 2375 2376

ENTRYPOINT ["dockerd-entrypoint.sh"]
CMD []

USER rootless

There is no VPNkit package for Debian/Ubuntu, so we need to install slirp4netns and use it. But, we get the same result = limiting resources still does not work. To run the above:

# docker run --rm -it --privileged --name dind-rootless -e TINI_SUBREAPER=1 -e DOCKER_TLS_CERTDIR="" -p 2375:2375 -e DOCKERD_ROOTLESS_ROOTLESSKIT_NET=slirp4netns -e DOCKERD_ROOTLESS_ROOTLESSKIT_MTU=65520 dind-rootless-debian /usr/local/bin/dockerd-entrypoint.sh --tls=false

Another problem for me though is that Kubernetes currently does not support cgroups v2 -- kubernetes/enhancements#2254

AkihiroSuda · 2021-10-18T18:08:11Z

limiting resources still does not work

Needs systemd

dg424 · 2021-10-18T18:17:41Z

Hi Akihiro,

I added systemd to the install list, but still no luck with resource limiting. Do I need to do some configuration of it as well ?

thaJeztah · 2021-10-18T18:37:31Z

Not exactly sure what the full intent is, but If systemd is needed inside the container, it likely needs to be running as PID1 (main process of the container). IIRC Rootless docker (started with systemd) requires docker to be run with a user systemd unit. If that's correct, you'll need a "privileged" systemd running as the container's main process, and after that, a user session that starts docker as a user-systemd service.

dg424 · 2021-10-18T18:46:50Z

Hi Sebastian,

The goal is to get a version of Docker Rootless with resource limiting working (https://docs.docker.com/engine/security/rootless/#limiting-resources). After chatting with Akihiro and Tianon, it seems like the only way to make this happen is with a Debian-based image (rather than the current Alpine image being used). Everything is setup as per the referenced documentation on the host side. The only issue now is getting Rootless Docker to do resource limiting.

thaJeztah · 2021-10-18T18:58:00Z

Right, but the reason Debian was mentioned, is because it requires a functional systemd (which Alpine doesn't provide). Systemd expects to be run as PID 1, so (for starters), the dockerd-entrypoint.sh cannot be used, as this is used to setup dockerd as the container's main process, not systemd.

Which is quite likely why Tianon's reply further up was;

It's possible, but it's definitely going to be complicated and likely going to be really fragile.

dg424 · 2021-10-18T19:03:17Z

Ok, so I assume that the documentation for limiting resources with rootless only applies to using the rootless daemon as a host daemon and not within DinD ?

thaJeztah · 2021-10-18T19:43:05Z

Limiting resources
Limiting resources with cgroup-related docker run flags such as --cpus, --memory, --pids-limit is supported only when running with cgroup v2 and systemd.

Its not necessarily "on the host" versus "docker in docker", needs systemd. Setting up everything in a container will be complex (and would almost be nearing creating a full system / VM-like container)

dg424 · 2021-10-18T20:11:41Z

Yes, I'm trying to figure out how they tested what is documented. I'm assuming rootless directly on the host or is it just in theory that it should work ?

thaJeztah added area/rootless Rootless mode kind/question version/20.10 labels Oct 6, 2021

AkihiroSuda changed the title ~~Cannot get cgroups v2 working with rootless container~~ [dind] Cannot get cgroups v2 working with rootless container Oct 6, 2021

natevw mentioned this issue Feb 3, 2022

cgroups v2 support - possible with systemd/freeipa-container ? freeipa/freeipa-container#429

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dind] Cannot get cgroups v2 working with rootless container #42910

[dind] Cannot get cgroups v2 working with rootless container #42910

dg424 commented Oct 5, 2021 •

edited by thaJeztah

AkihiroSuda commented Oct 6, 2021

AkihiroSuda commented Oct 6, 2021

ddag commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021 •

edited

tianon commented Oct 6, 2021 via email

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021 •

edited

dg424 commented Oct 18, 2021 •

edited

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021 •

edited

dg424 commented Oct 18, 2021 •

edited

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

AkihiroSuda commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

[dind] Cannot get cgroups v2 working with rootless container #42910

[dind] Cannot get cgroups v2 working with rootless container #42910

Comments

dg424 commented Oct 5, 2021 • edited by thaJeztah

AkihiroSuda commented Oct 6, 2021

AkihiroSuda commented Oct 6, 2021

ddag commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021 • edited

tianon commented Oct 6, 2021 via email

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021

tianon commented Oct 6, 2021

dg424 commented Oct 6, 2021

dg424 commented Oct 6, 2021 • edited

dg424 commented Oct 18, 2021 • edited

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021 • edited

dg424 commented Oct 18, 2021 • edited

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

AkihiroSuda commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

thaJeztah commented Oct 18, 2021

dg424 commented Oct 18, 2021

dg424 commented Oct 5, 2021 •

edited by thaJeztah

dg424 commented Oct 6, 2021 •

edited

dg424 commented Oct 6, 2021 •

edited

dg424 commented Oct 18, 2021 •

edited

dg424 commented Oct 18, 2021 •

edited

dg424 commented Oct 18, 2021 •

edited