Homelab: Kubernetes Home Cluster - Infrastructure

This repository contains the infrastructure as code (IaC) for the home-cluster using Pulumi.

---

Requirements

• NodeJS, and yarn
• Pulumi
• k0sctl

Creating the Infrastructure

To create the infrastructure and deploy the cluster, a Pulumi Stack with the correct configuration needs to exists.

The stack can be deployed via:

yarn install
yarn build; pulumi up

Destroying the Infrastructure

The entire infrastructure can be destroyed via:

yarn install
yarn build; pulumi destroy

Environment Variables

To successfully run, and configure the Pulumi plugins, you need to set a list of environment variables. Alternatively, refer to the used Pulumi provider's configuration documentation.

• CLOUDSDK_CORE_PROJECT: the Google Cloud (GCP) project
• CLOUDSDK_COMPUTE_REGION the Google Cloud (GCP) region
• GOOGLE_APPLICATION_CREDENTIALS: reference to a file containing the Google Cloud (GCP) service account credentials
• DOPPLER_TOKEN: the Doppler access token (not a project token!)
• GITHUB_TOKEN: the GitHub Personal Access Token (PAT)
• PROXMOX_VE_USERNAME: the Proxmox username
• PROXMOX_VE_PASSWORD: the Proxmox password
• PROXMOX_VE_ENDPOINT: the endpoint to connect to Proxmox
• PROXMOX_VE_INSECURE: turn on/off insecure connections to Proxmox

---

Configuration

The following section describes the configuration which must be set in the Pulumi Stack.

Attention: do use Secrets Encryption provided by Pulumi for secret values!

Bucket Identifier

bucketId: the bucket identifier to upload assets to

Cluster

The cluster exists from created Proxmox KVM servers.

cluster:
  name: the cluster name
  nodes: a map of servers to create in Promxox
    <NODE_NAME>:
      cpu: the CPU allocation
      diskSize: the disk size to use
      memory: memory configuration (enables or disables ballooning automatically)
        min: the minimum memory to assign
        max: the maximum memory to assign
      host: the Proxmox host to create the node on
      ipv4Address: the internal IPv4 address
      ipv6Address: the internal IPv6 address (optional)
      roles: a list of all k0s roles (the first one is chosen!)
      labels: a map of Kubernetes node labels to apply

Google Cloud (GCP)

Flux deployed applications can reference secrets being encrypted with sops. We need to specify, and allow access to this encryption stored in Google KMS.

gcp:
  project: the GCP project to create all resources in
  encryptionKey: references the sops encryption key
    cryptoKeyId: the CryptoKey identifier
    keyringId: the KeyRing identifier
    location: the location of the key

Network

General configuration about the local network.

network:
  domain: the internal DNS domain
  ipv4:
    cidrMask: the CIDR mask of the internal network
    enabled: enables IPv4 networking
    gateway: the IPv4 gateway
  ipv6:
    cidrMask: the CIDR mask of the internal network
    enabled: enables IPv6 networking
    gateway: the IPv6 gateway
  nameservers: a list of all nameservers to set (IPv4, IPv6)

Proxmox VE (pve)

General configuration about the Proxmox environment.

Attention: you must download the specifief imageName to each Proxmox host!

pve:
  cpuType: the default CPU type to assign to machines
  imageName: the reference to the locally installed image
  localStoragePool: the storage pool used for snippets
  networkBridge: the network bridge to use for server connectivity
  storagePool: the storage pool used for machine disks

k0s

k0s is used as the Kubernetes distribution. Additionally, Cilium as the CNI is installed.

k0s:
  version: the k0s Kubernetes version
  cilium:
    enabled: enables deployment of cilium
    version: the version of the cilium Helm chart to deploy initially

Username

username: the username to use for interacting with the servers

SSH

The SSH key needs to be specified especially for the external nodes, and for all operations to be able to connect to these servers.

ssh:
  privateKey: the SSH private key to provision/use
  publicKey: the SSH public key to use

UFW

ufw:
  enabled: turns on/off provisioning of the UFW in the inventory.yml file

---

Continuous Integration and Automations

• GitHub Actions are linting, and verifying the code.
• Renovate Bot is updating NodeJS packages, and GitHub Actions.