Ansible Collection - Rkub

Ansible Collection to deploy a RKE2 cluster in airgap mode with Rancher, Longhorn and Neuvector.

Description

This Ansible collection will install in airgap environnement RKE2 (one controler and several workers, currently no HA):

Current develop - Ansible Collection Rkub 1.0.3 include:

• RKE2 1.27.10 - Security focused Kubernetes (channel stable)

• Kube-vip 0.7.0 - Virtual IP and load balancer

• Cert-manager 1.14.1 - Certificate manager

• Rancher 2.8.1 - Multi-Cluster Kubernetes Management

• Longhorn 1.6.0 - Unified storage layer

• Neuvector 2.7.2 - Kubernetes Security Platform

This Project is mainly inspired from Clemenko/rke_airgap_install. I tried it and like the idea but I was frustrated with Shell scripting limitations. So I decided to rewrite it in Ansible.

With Ansible:

• Idempotency: can be relaunch multiple time.

• User agnostic: can be launch by any user (with sudo rights).

• OS agnositc: can be launch on any Linux systems (at least for the package build, for the install part, it depends on your participation 😸)

Add-on from my part:

• Some flexibility about path with the possibility to build and install on a choosen path.

• Admin user (by default 'kuberoot') on first controller node with some admin tools (k9s, helm).

• Nerdctl as complement of containerd to handle oci-archive.

• K9S on first controller for admin purpose.

• Uninstall playbook to cleanup (and maybe reinstall if needed).

• Collection Released, so possibilty to get back to older versions.

Prerequisites

• Linux Host as a package builder (can be a VM or your WSL). Count 10G of free space in the build directory of your package builder.

• An Ansible Controler, can be the same host for ansible and for building package, at your convenience...

• A minimum of 2 hosts RHEL-like (2 vCPU and 8G of RAM) for the cluster RKE2 with 80G at least on target directory.

Getting started

1. Preparation steps:

• Clone the main branch of this project to a machine with an internet access: git clone -b main https://github.com/MozeBaltyk/Rkub.git

• Execute make prerequis to install all prerequisites defined in meta directory.

• Complete directory inside ./plugins/inventory/hosts.yml.

2. Build your package by running (works on Debian-like and Redhat-like and it targets localhost):

ansible-playbook playbooks/tasks/build.yml                    # All arguments below are not mandatory
-e "dir_build=$HOME/rkub"                                     # Directory where to upload everything (count 30G)
-e "package_name=rkub.zst"                                    # Name of the package, by default rkub.zst
-e "archive=true"                                             # Archive tar.zst true or false (default value "true")
-e "stable=false"                                             # Stable channels or defined version in Rkub collection (default value "false")
-e "el=9"                                                     # RHEL version (take default value from localhost if OS is different from RedHat-like take value "8")
-e "all=false"                                                # if you want to install all components kubevip,longhorn,rancher,neuvector (default value "false")
-e "kubevip=true longhorn=true rancher=true neuvector=true"   # which extras components you want to add to package (default value from var 'all')
-u admin -Kk                                                  # Other Ansible Arguments (like -vvv)

3. Push your package to first controler:

ansible-playbook playbooks/tasks/upload.yml        # All arguments below are not mandatory
-e package_path=/home/me/rkub.zst                  # Will be prompt if not given in the command
-e dir_target=/opt/rkub                            # Directory where to sync and unarchive (by default /opt/rkub, count 50G available)
-u admin -Kk                                       # Other Ansible Arguments (like -vvv)

4. Start installation:

ansible-playbook playbooks/tasks/install.yml       # All arguments below are not mandatory
-e dir_target=/opt/rkub                            # Dir on first master where to find package unarchive by previous task (by default /opt/rkub, count 50G available)
-e domain="example.com"                            # By default take the host domain from master server
-u admin -Kk                                       # Other Ansible Arguments (like -vvv)

5. Deploy Rancher:

ansible-playbook playbooks/tasks/rancher.yml       # All arguments below are not mandatory
-e dir_mount=/mnt/rkub                             # NFS mount point, by default value is /mnt/rkub
-e domain="example.com"                            # Domain use for ingress, by default take the host domain from master server
-e password="BootStrapAllTheThings"                # Default password is "BootStrapAllTheThings"
-u admin -Kk                                       # Other Ansible Arguments (like -vvv)

6. Deploy Longhorn:

ansible-playbook playbooks/tasks/longhorn.yml      # All arguments below are not mandatory
-e dir_mount=/mnt/rkub                             # NFS mount point, by default value is /mnt/rkub
-e domain="example.com"                            # Domain use for ingress, by default take the host domain from master server
-e datapath="/opt/longhorn"                        # Longhorn Path for PVC, by default equal "{{ dir_target }}/longhorn".
                                                   # The best is to have a dedicated LVM filesystem for this one.
-u admin -Kk                                       # Other Ansible Arguments (like -vvv)

7. Deploy Neuvector

ansible-playbook playbooks/tasks/neuvector.yml     # All arguments below are not mandatory
-e dir_mount=/mnt/rkub                             # NFS mount point, by default value is /mnt/rkub
-e domain="example.com"                            # Domain use for ingress, by default take the host domain from master server
-u admin -Kk                                       # Other Ansible Arguments (like -vvv)

8. Bonus:

With make command, all playbooks above are in the makefile. make alone display options and small descriptions.

# Example with make
make install                                       # All arguments below are not mandatory
ANSIBLE_USER=admin                                 # equal to '-u admin'
"OPT=-e domain=example.com -Kk"                    # redefine vars or add options to ansible-playbook command

Container methode

1. This is a custom script which imitate Execution-Environement:

• make ee-container will load an UBI-8 image and execute inside make prerequis

• make ee-exec Run image with collection and package zst mounted inside. Launch playbook or make command as described above.

All prerequisites are set in folder meta and meta/execution-environment.yml. So it's possible to use ansible-builder (though not tested yet).

Some details

I favored the tarball installation since it's the one the most compact and also leave an tar.zst on all nodes.

Build have for purpose to create a tar zst with following content:

rkub
├── helm          # all helm charts
├── images        # all images
│   ├── cert
│   ├── longhorn
│   ├── neuvector
│   ├── rancher
│   └── registry
├── rke2_1.26.11  # RKE2 binaries
└── utils         # utilities packages downloaded with arkade

upload push the big monster packages (around 7G) and unarchive on first node on chosen targeted path.

install RKE2 (currently only one master) with:

• An admin user (by default kuberoot) on first master with some administation tools like k9s kubectl or helm.
• Master export NFS with all the unarchive content + registry content
• Workers mount the NFS to get above content
• A minimal registry is deploy on each nodes pointing to the NFS mount and responding to localhost:5000
• Nerdctl as complement to containerd and allow oci-archive
• Firewalld settings if firewalld running

deploy keeping this order, Rancher, Longhorn, Neuvector

• Those are simple playbooks which deploy with helm charts
• It use the default ingress from RKE2 Nginx-ingress in https (currently Self-sign certificate)
• Rancher need Certmanager, So it deploy first Certmanager

Roadmap

Milestones:

• More install customization and options

• HA masters with kubevip

• Add a option to chooce by url mode or airgap mode

Improvments:

• Improve collection to run as true collection

• CI

Acknowledgements

Special thanks to 📢

• Clemenko, for the idea Clemenko/rke_airgap_install.

References:

• Clemenko/rke_airgap_install

• rancherfederal/RKE2-ansible

• lablabs/ansible-role-rke2

• rancher/RKE2

Get the latest stable version:

## RKE2
curl -s https://raw.githubusercontent.com/rancher/rke2/master/channels.yaml | yq -N '.channels[] | select(.name == "stable") | .latest'

## K3S
curl -s https://raw.githubusercontent.com/k3s-io/k3s/master/channel.yaml | yq -N '.channels[] | select(.name == "stable") | .latest'

Repo Activity

Project status

Still on developement