pivot_root(".", ".") fails with EINVAL at src/lxc/conf.c:lxc_pivot_root:1598 on RH9.2 on Alpine Edge

[rickyrockrat]

This happens on an Alpine edge X86_64 system. The native lxc on the alpine host (lxc-5.0.3) runs this command successfully with the exact same rootfs.

Alpine Linux Edge x86_64 kernel user@hostname:/500g/src/rhel$ uname -a
Linux hostname 6.1.62-0-lts #1-Alpine SMP PREEMPT_DYNAMIC Thu, 09 Nov 2023 07:17:35 +0000 x86_64 GNU/Linux

Running inside a Red Hat 9.2 Chroot with the following mounts:

none on /proc type proc (rw,relatime)
none on /sys type sysfs (rw,relatime)
none on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=16463210,mode=755,inode64)
none on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
none on /sys/fs/cgroup type tmpfs (rw,relatime,inode64)
none on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
none on /dev/shm type tmpfs (rw,relatime,inode64)
none on /dev/mqueue type mqueue (rw,relatime)
none on /sys/kernel/security type securityfs (rw,relatime)
none on /sys/kernel/debug type debugfs (rw,relatime)
none on /sys/fs/pstore type pstore (rw,relatime)

brctl show (in rh9.2 chroot)
bridge name bridge id STP enabled interfaces
lxcbr0 8000.000000000000 no

Built source code from lxc-5.0.2.tar.gz on same Red Hat 9.2 chroot.

Trying to run a minimal alpine Linux chroot with lxc and the following config:

lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:dd:d1:ff
lxc.rootfs.path = /containers/alp2

/containers/alp2 is created with:
cd $dir
tar -xf /path/to/apk-tools-static-*
cd ..
sudo $dir/sbin/apk.static --arch $arch -X $repo -U --allow-untrusted --root $dir --initdb add alpine-base bash

lxc-execute almost runs, but it gets to the line where it tries to call pivot_root(".","."), and fails with EINVAL.

I ran this command in the RH9.2 chroot to create the attached files:
strace -s100 -o /tmp/lxcexe.log -f lxc-execute -l trace -o /tmp/lxcexe-trace.log -n gpp1 /bin/ls

lxc-execute: gpp1: ../src/lxc/sync.c: sync_wait: 34 An error occurred in another process (expected sequence number 4)
lxc-execute: gpp1: ../src/lxc/start.c: __lxc_start: 2107 Failed to spawn container "gpp1"
lxc-execute: gpp1: ../src/lxc/tools/lxc_execute.c: main: 224 Failed run an application inside container

I'm sure it's something silly I didn't do in the RH9.2 chroot, but I'm a little stumped right now.
It seems related to #61

thanks in advance.

[mihalicyn]

Hi @rickyrockrat !

When you are doing chroot you effectively changing current's task VFS root dentry to some another one.
chroot does not care if a new "root" dentry is a mountpoint or not. While pivot_root does care! You need to have (in LXC case) container's rootfs to be a mountpoint! Likely, in your case this condition is not satisfied. What I can suggest you (which can theoretically help) is to create a bindmount like this mount --bind /your/chroot/directory /your/chroot/directory and then chroot /your/chroot/directory. I'm not sure if you won't meet any troubles somewhere else, but it makes sense to try :)