This repo contains Kafka, Kafka Connect, Hazelcast and Prometheus containers. JMX metrics from Kafka Connect service are exposed with the prometheus jmx exporter. Hazelcast built-in JMX Prometheus Agent is also tested for JMX and JMX Hazelcast metrics.
Kafka Connect service is configured as FileStream Source Connector using a Kafka base image with the prometheus jmx exporter installed.
JMX options can be customized using the KAFKA_JMX_OPTS and KAFKA_OPTS environment variables.
Prometheus metrics will be exported on port 9090. The jmx metrics that are exported are those that match the patterns in the jmx_exporter_config.yml config.
- Build the custom docker image locally:
docker-compose build - Run
docker-compose up -dwill run the containers in background - Attach a console consumer to the topic to consume when new messages are published to it:
docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic simple-connect --from-beginning
- Check Prometheus dashboard for Kafka Connect JMX metrics:
localhost:9090