BOOTiful Microservices

Spring-boot based Microservice platform with

• Distributed messaging (Apache Kafka)
• Distributed logging (Fluentd, OpenDistro Elasticsearch)
• Distributed tracing (Jaeger)
• Metrics collection and visualization (Prometheus, Loki, Grafana)
• Rest endpoint documentation using Swagger and OpenAPI
• AWS lambda function running locally in Localstack

CI and CD

• Jenkins CICD pipeline is included in every service that will build and deploy the service
• Ability to switch build profile at build time using Maven profiles for either
  • Container platform
  • AWS Lambda

Pre-requisite

• JDK 11 compiler
• Maven > 3.5.0
• Docker > 1.18 (Docker desktop on Windows and MacOS will work)
• Docker compose (latest version. Comes with Docker desktop)
• Kubernetes is turned on in Docker Desktop

Understanding components in this project

Spring boot pom parent (anair-pom-parent)

• Parent pom that manages commonly used dependencies and plugins.
• All services in this project will extend from this parent pom.
• Read more...

Spring boot Rest Maven archetype (anair-rest-archetype)

• Template project that can be used to generate a Restful spring boot application with sample java classes, properties
• Read more...

Spring boot services

• Admin console
  • Spring boot services admin console
• Rest Service
  • Exposes a rest endpoint
  • Publishes a message on a Kafka topic
  • Invoke a GET grpc endpoint
• Kafka client
  • Consumes message from the Kafka topic published by Service A
• Protobuf Service
  • Expose grpc endpoint producing protobuffer content when invoked by Service A
• Lambda Function
  • AWS lambda Fn invoked using SQS and/or CloudWatch event. Creates a S3 file and sends a message to an outbound SQS queue

Helm project for anair-service-rest

• Helm scripts to deploy service-rest to k8s cluster. Read more

Logging

Publish application and system logs to a centralized server.

• Logs from applications and systems are collected by Fluentd
  • Review fluentd conf at logging/fluentd/conf/fluent.conf that has elasticsearch and loki outputs
• Fluentd publishes the logs to OpenDistro Elasticsearch server and Loki
• Visualize logs in Kibana
• Visualize logs in Grafana using Loki datasource

Metrics

• Read more on Metrics...

Infrastructure services

The spring boot microservices integrates with the following infrastructure components:

• Jaeger (Distributed tracing)
• Prometheus (Time-series metrics collector)
• Loki (Index free log collector just like Prometheus)
• Grafana (View Prometheus metrics and Loki logs)
• OpenDistro Elasticsearch and Kibana (Distributed logging)
• AWS Localstack

Middleware services

The spring boot microservices integrates with the following middleware components:

• Apache Zookeeper (Distributed coordination for Kafka)
• Apache Kafka (Distributed messaging)

Running microservices

In a Docker environment

1. Navigate to this project
2. Review docker-compose.yml and docker-compose-infra.yml files
3. Run mvn clean package on anair-service-rest, anair-service-kafka, anair-service-proto, anair-service-admin. This has to be done anytime code is changed and has to be deployed in docker.
4. Run docker-compose -f docker-compose-infra.yml up to start infrastructure services:
   • Jaeger
   • Elasticsearch
   • Kibana
   • Prometheus
   • Prometheus node exporter
   • cAdvisor
   • Loki
   • Grafana
   • Localstack (AWS stack)
5. Run docker-compose -f docker-compose-middleware.yml up to start middleware services:
   • Fluentd
   • Zookeeper
   • Kafka
6. Run docker-compose up to start microservices:
   • Spring boot admin
   • Rest Service
   • Kafka Service
   • Protobuf Service
7. Verify that the below mentioned URls are accessible

Urls

• Jaeger
• Prometheus
• Grafana
  • Login initially as admin/admin. If asked to enter new password, skip or change.
• cAdvisor
• Kibana
  • Login as admin/admin
• Swagger - Rest Service
• Swagger - Protobuf Service
• Spring boot admin
  • Login as admin/admin
• Localstack

Executing service endpoints

• In Service A swagger page, try out /publish/{userId} endpoint. This will execute a kafka and grpc transaction. Check logs of all 3 services. View the trace graph in Jaeger UI.