Kubernetes-CICD-Monitoring Repository

Description: This GitHub repository contains all the necessary configuration files and manifests to set up a Kubernetes cluster with Continuous Integration/Continuous Deployment (CI/CD) pipelines and monitoring components. The repository is designed to work with Minikube, a local Kubernetes cluster for development and testing purposes.

Key Features:

• Manifests for Kubernetes Cluster: Includes Kubernetes manifests to create the core cluster components.
• CI/CD Pipeline: A Jenkins-based CI/CD pipeline is integrated into the cluster for automated application deployment.
• Gitea Example: Demonstrates the setup of Gitea, a self-hosted Git service. Gitea is a lightweight Git service that can be used for code version control.
• Monitoring Stack: Sets up monitoring using Prometheus and Grafana to monitor the cluster's health and performance.

Getting Started:

Requirenments:

• Docker (tested on v20.10.20)
• kubectl (tested on v1.29.2)
• gh cli tool (tested on v2.45.0)
• minikube (tested on v1.32.0)

Optional requirements:

• ngrok (if you want to use webhook)
• k9s (to monitor cluster state)

Optional pre step: If you want to use ngrok and github webhook configure ngrok account and get unic domain name (you can do it here https://dashboard.ngrok.com/cloud-edge/domains). Also you need to configure free access for GitHub via Ngrok Edge and setup oAuth to improve secutiry of your endpoint. To start ngrok tunel on background execute:

 ngrok tunnel --label edge=<past your edge label here> $(minikube ip):8080>/dev/null &

To start Kubernetes cluster to create CI/CD, follow these steps:

1. Autorize to gh-cli via $gh auth login
2. Start minikube (I recommend to use --memory 8192 --cpus 4 unless build can fail (it was when I tested it on Gitea build))

   minikube start --memory 8192 --cpus 4 --driver=docker

3. Connect the Docker environment of your host machine to Minikube:

   eval $(minikube docker-env)

4. Configure the jenkinsCICD/jenkinsconf.yaml and jenkinsCICD/jobs/giteaCICD/config files:

   • Set the URL to your Git repository in the job configurations located at jenkinsCICD/jobs/giteaCICD/config. (NOTE: in your repository should be Jenkins file in root of project. Example for Gitea is in /Jenkinsfile-example file, also there should be kubernetes manifests for raise of infrasracture and Dockerfile or build gitea container)

5. Build the Jenkins Docker image:

   docker build -t jenkins:jcasc ./jenkinsCICD

6. Deploy the infrastructure components to Minikube:

   export NGROK_DOMAIN=<paste here your ngrok domain> #ignore if you don't need gh webhook
   export GH_TOKEN=<paste here your gh token>
   export TG_TOKEN=<paste here your tg bot token>
   envsubst < ./jenkinsCICD/jenkinsK8sCICD.yaml |kubectl apply -f -

7. Create and get Jenkins service to have access to Jenkins WebUI :

   kubectl expose deployment jenkins-deployment --type=LoadBalancer --external-ip=$(minikube ip) --name jenkins-service
   kubectl get services

8. Wait for Jenkins to scan your repository, perform artifact builds, and create Docker images.
9. After Jenkins completes its tasks, it will deploy the Gitea infrastructure with a PostgreSQL database in the Minikube cluster.
10. To expose Gitea to external ip you also should execute (but only after success creation of gitea and postgresql pods):

   kubectl expose deployment gitea-deployment --type=LoadBalancer --namespace=gitea-app --external-ip=$(minikube ip) --name gitea-service

Monitoring Stack Components: For the monitoring components, I used manifests from the following repositories:

• kubernetes-grafana
• kube-state-metrics-configs
• kubernetes-prometheus

To implement monitoring folow this steps:

1. Apply all manifests for Prometheus

   kubectl apply -f ./monitoring/kubernetes-prometheus -R -n monitoring

2. Apply all manifests for Grafana

   kubectl apply -f ./monitoring/kubernetes-grafana -R -n monitoring

3. Expose Grafana to external ip

kubectl expose deployment grafana --type=LoadBalancer --namespace=monitoring --external-ip=$(minikube ip) --name grafana-service  --port=5050 --target-port=3000

Note: This repository provides a comprehensive example of setting up a Kubernetes cluster for CI/CD and monitoring purposes. Customize the configurations to suit your specific project requirements.

Contributions and Issue Reporting:

Feel free to contribute to this repository or report any issues you encounter. Your contributions and feedback are highly appreciated.