Monitoring the JVM on Kubernetes

Pre-requisites

Tools:

• kubectl
• kubectx
• minikube
• helm 2

Setup

NOTE: these instructions have been tested on MacOS Catalina 10.5.

To bootstrap your minikube cluster simply run the bootstrap.sh script.

This will enable the following capabilities:

• A Kubernetes cluster on v.1.15.9
• Helm 2
• Prometheus operator (via helm)

Expose the Prometheus services on your local host

We'll use port-forwarding to expose the services running on the cluster to your local machine.

The port-forwading.sh script will do this for us.

Deploy an app

The application we will deploy is called Chaos Kraken and it is designed to simulate various failure scenarios.

Chaos Kraken is an open source project created at Auto Trader UK. It is used to test and validate various behaviours of their Kubernetes based delivery platform.

./deploy-chaos-kraken.sh

Troubleshooting

Check your prometheus config

kubectl -n monitoring get secret prometheus-prometheus-prometheus-oper-prometheus -ojson | jq -r '.data["prometheus.yaml.gz"]' | base64 -d | gunzip

Grafana username & password

The username & password for Grafana are stored in a secret:

kubectk get secret -n monitoring prometheus-grafana -ojson | jq -r '.data["admin-user"]' | base64 -d
kubectl get secret -n monitoring prometheus-grafana -ojson | jq -r '.data["admin-password"]' | base64 -d

The defaults are admin/prom-operator and as we are running this locally via Minikube we won't worry about making it more secure.