Including cryptsetup into longhorn container image

[dgrechka]

Hi all,

Why does longhorn requires to install cryptsetup on the k8s nodes to enable encryption? Why can't we instead include cryptsetup into the engine or manager image (not sure in which it is required)?
For some setups, like k3os, it is hard to install dependencies like these on the nodes.

[joshimoo]

@dgrechka as a stop gap you can try deploying a privileged deamon set, with HostMNT, HostNET and HostIPC as well as correctly setup bind mounts (make crypytsetup available as part of path), that way when the csi plugin nsenters into the host namespace. It will execute your cryptsetup binary as long as it is part of the path. REF: https://github.com/longhorn/longhorn-manager/blob/fade760c17abe2aec1a23bf8a584f8c612e20675/csi/crypto/luks.go#L53

This deamonset can also be used to load/make available the dm_crypt kernel module.

[dgrechka]

@joshimoo thank you for the idea.
Could you please elaborate on what the daemon set should do?
Should it "deploy" the cryptsetup to the host?

I've tried the following:

1. Prepared image with cryptsetup that works.
   Dockerfile

FROM alpine:latest
RUN apk add cryptsetup
RUN rm /etc/profile.d/*
# copy the PATH extender
COPY cryptsetup.env.sh "/etc/profile.d/cryptsetup.env.sh"
COPY deploy.sh "/tmp/deploy.sh"
CMD ["/bin/sh","-c","--","source /tmp/deploy.sh; while true; do sleep 30; done;" ]

2. Run the following script in the daemonset (deploy.sh):

#!/bin/sh

# required mounts (container -> host):
# /mnt/host.profile.d/ -> /etc/profile.d # used to extend $PATH
# /mnt/host.cryptsetup/ -> /var/lib/cryptsetup # used to provide CLI
# /mnt/host.lib/ -> /lib/ # used to deploy libs

# deploying PATH extender
cp -v /etc/profile.d/*.sh /mnt/host.profile.d/

# deploying CLI
rm /mnt/host.cryptsetup/*
# see https://pkgs.alpinelinux.org/contents?file=&path=&name=cryptsetup&repo=main&arch=x86_64
cp /sbin/cryptsetup /mnt/host.cryptsetup/
cp /sbin/cryptsetup-reencrypt /mnt/host.cryptsetup/
cp /sbin/integritysetup /mnt/host.cryptsetup/
cp /sbin/veritysetup /mnt/host.cryptsetup/

# deploying libs
#rm /mnt/host.lib/libcryptsetup*
cp /lib/libcryptsetup* /mnt/host.lib/

3. Deploy the daemonset with the following configuration

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: cryptsetup-deployer
  namespace: longhorn-system
  labels:
    k8s-app: cryptsetup-deployer
spec:
  selector:
    matchLabels:
      name: cryptsetup-deployer
  template:
    metadata:
      labels:
        name: cryptsetup-deployer
    spec:
      containers:
      - name: cryptsetup-deployer
        image: dgrechka/cryptosetup_deployer:latest
        imagePullPolicy: Always
        securityContext:
          privileged: true
        volumeMounts:
        - name: sbin
          mountPath: /mnt/host.cryptsetup/
          mountPropagation: Bidirectional   
        - name: profile-d
          mountPath: /mnt/host.profile.d/
          mountPropagation: Bidirectional   
        - name: lib
          mountPath: /mnt/host.lib/
          mountPropagation: Bidirectional   
      volumes:
      - name: sbin
        hostPath:
          path: /var/lib/cryptsetup/
      - name: profile-d
        hostPath:
          path: /etc/profile.d/
      - name: lib
        hostPath:
          path: /lib/

When I SSH to node and run the cryptsetup I got the following:

k8s-charlie [~]# cryptsetup
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_token_external_disable: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_reencrypt_run: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_logf: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_activate_by_token_pin: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_header_is_detached: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_token_external_path: symbol not found
Error relocating /var/lib/cryptsetup/cryptsetup: crypt_dump_json: symbol not found

It seems that the dependencies of cryptsetup are not fully available.

Could you advice how to deploy cryptsetup more correctly? Maybe I did not get your idea in full.
Thanks.