kind-cluster

Create local Kubernetes clusters using Docker with KinD

Requirements

• Docker
• kind
• kubectl
• helm
• curl
• jq
• base64

Help

make help

help                               - List available tasks
install-all                        - Install all (kind k8s cluster, Nginx ingress, MetaLB, demo workloads)
install-all-no-demo-workloads      - Install all (kind k8s cluster, Nginx ingress, MetaLB)
create-cluster                     - Create k8s cluster
export-cert                        - Export k8s keys(client) and certificates(client, cluster CA)
k8s-dashboard                      - Install k8s dashboard
nginx-ingress                      - Install Nginx ingress
metallb                            - Install MetalLB load balancer
deploy-app-nginx-ingress-localhost - Deploy httpd web server and create an ingress rule for a localhost (http://demo.localdev.me:80/), Patch ingress-nginx-controller service type -> LoadBlancer
deploy-app-helloweb                - Deploy helloweb
deploy-app-golang-hello-world-web  - Deploy golang-hello-world-web app
deploy-app-foo-bar-service         - Deploy foo-bar-service app
delete-cluster                     - Delete K8s cluste

Install all (kind k8s cluster, Nginx ingress, MetaLB, demo workloads)

./scripts/kind-install-all.sh

Or you can install each component individually

Create k8s cluster

./scripts/kind-create.sh

Export k8s keys(client) and certificates(client, cluster CA)

./scripts/kind-create.sh

Script creates:

• client.key
• client.crt
• client.pfx
• cluster-ca.crt

Install k8s dashboard

Install k8s dashboard

./scripts/kind-add-dashboard.sh

Script creates file with admin-user token

• dashboard-admin-token.txt

Launch k8s Dashboard

v3.0.0-alpha0

helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard
kubectl apply -n kubernetes-dashboard -f ./k8s/dashboard-admin.yaml
export dashboard_admin_token=$(kubectl get secret -n kubernetes-dashboard admin-user-token -o jsonpath="{.data.token}" | base64 --decode)
echo "${dashboard_admin_token}" > dashboard-admin-token.txt
kubectl config set-credentials cluster-admin --token=${dashboard_admin_token}
echo "Dashboard Token: ${dashboard_admin_token}"

export POD_NAME=$(kubectl get pods -n kubernetes-dashboard -l "app.kubernetes.io/name=kubernetes-dashboard,app.kubernetes.io/instance=kubernetes-dashboard" -o jsonpath="{.items[0].metadata.name}")
kubectl -n kubernetes-dashboard port-forward $POD_NAME 8443:8443
xdg-open "https://localhost:8443"

# helm delete kubernetes-dashboard --namespace kubernetes-dashboard
# kubectl delete clusterrolebinding --ignore-not-found=true kubernetes-dashboard
# kubectl delete clusterrole --ignore-not-found=true kubernetes-dashboard

v2.x

# kill kubectl proxy if already running
pkill -9 -f "kubectl proxy"
# start new kubectl proxy
kubectl proxy –address=’0.0.0.0′ –accept-hosts=’^*$’ &
# copy admin-user token to the clipboard
cat ./dashboard-admin-token.txt | xclip -i
# open dashboard
xdg-open "http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/" &

In Dashboard UI select "Token' and Ctrl+V

Install Nginx ingress

./scripts/kind-add-ingress-nginx.sh

Install MetalLB load balancer

./scripts/kind-add-metallb.sh

Install NFS & nfs-subdir-external-provisioner to achieve RWX

• NFS Server and Client on Ubuntu 22.04
• Dynamic NFS Provisioning in Kubernetes Cluster
• Kubernetes – Setup Dynamic NFS Storage Provisioning – helm
• Mounting Volume with RWX mode in KIND Cluster using NFS

sudo apt install -y nfs-kernel-server nfs-common

sudo mkdir -p /mnt/k8s_nfs_storage
sudo chown -R nobody:nogroup /mnt/k8s_nfs_storage
sudo chmod 777 /mnt/k8s_nfs_storage

get your host IP

hostname -I | awk '{print $1}'

$ 192.168.1.27

let's allow any IP * (or you whole subnetwork 192.168.1.0/24)

sudo vi /etc/exports

/mnt/k8s_nfs_storage *(rw,sync,no_subtree_check)

sudo exportfs -a
sudo systemctl restart nfs-kernel-server
sudo systemctl status nfs-kernel-server

# add firewall rules
# sudo ufw status
# sudo ufw allow from 192.168.1.27 to any port nfs
sudo ufw allow nfs
sudo ufw disable
sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
Nginx HTTP                 ALLOW       Anywhere                  
Nginx Full                 ALLOW       Anywhere                  
22/tcp                     DENY        Anywhere                  
2049                       ALLOW       192.168.1.27              
Nginx HTTP (v6)            ALLOW       Anywhere (v6)             
Nginx Full (v6)            ALLOW       Anywhere (v6)             
22/tcp (v6)                DENY        Anywhere (v6) 

mount it test if it worked

sudo mkdir -p /mnt/nfs_clientshare/
sudo mount -t nfs 192.168.1.27:/mnt/k8s_nfs_storage /mnt/nfs_clientshare/
sudo umount -f -l /mnt/nfs_clientshare/

Install the nfs-subdir-external-provisioner

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner

docker pull registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
kind load docker-image registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
helm install -n nfs-provisioning --create-namespace nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server=192.168.1.27 --set nfs.path=/mnt/k8s_nfs_storage

kubectl get all -n nfs-provisioning
kubectl get sc -n nfs-provisioning

kubectl create -f ./k8s/nfs/pvc.yaml
kubectl get pv,pvc -n nfs-provisioning
kubectl create -f ./k8s/nfs/pod.yaml
kubectl get pods -n nfs-provisioning
kubectl exec --stdin --tty -n nfs-provisioning test-pod -- /bin/sh

Deploy demo workloads

Deploy httpd web server and create an ingress rule for a localhost http://demo.localdev.me:80/

./scripts/kind-deploy-app-nginx-ingress-localhost.sh

Deploy helloweb

./scripts/kind-deploy-app-helloweb.sh

Deploy golang-hello-world-web

./scripts/kind-deploy-app-golang-hello-world-web.sh

Deploy foo-bar-service

./scripts/kind-deploy-app-foo-bar-service.sh

Deploy Prometheus

Add prometheus and stable repo to local helm repository

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add stable https://charts.helm.sh/stable
helm repo update

Create namespace monitoring to deploy all services in that namespace

kubectl create namespace monitoring

Install kube-prometheus stack

helm template kind-prometheus prometheus-community/kube-prometheus-stack --namespace monitoring \
--set prometheus.service.nodePort=30000 \
--set prometheus.service.type=LoadBalancer \
--set grafana.service.nodePort=31000 \
--set grafana.service.type=LoadBalancer \
--set alertmanager.service.nodePort=32000 \
--set alertmanager.service.type=LoadBalancer \
--set prometheus-node-exporter.service.nodePort=32001 \
--set prometheus-node-exporter.service.type=LoadBalancer \
> ./k8s/prometheus.yaml

kubectl apply -f ./k8s/prometheus.yaml
kubectl --namespace monitoring get pods -l release=kind-prometheus

Delete kube-prometheus stack

kubectl delete -f ./k8s/prometheus.yaml

Delete k8s cluster

./scripts/kind-delete.sh