Introduction to Spring Cloud Stream Kafka

Introduction to Spring Cloud Stream Kafka

Spring Cloud Stream Kafka (Part1)

Documentation (Part 1)

Project Source Code

Spring Cloud Stream Kafka (Part2) Binders

Order Example

Key features:

• spring boot (2.4.13)
• Spring cloud Stream kafka (Hoxton.SR12)
• Binders @EnableBinding

Documentation (Part 2) Binders

Project Source Code

⚠ NOTE: Starting with spring-cloud-stream-binder-kafka/spring-cloud-starter-stream-kafka 3.0, when spring.cloud.stream.binding.<name>.consumer.batch-mode is set to true, all of the records received by polling the Kafka Consumer will be presented as a List<?> to the listener method. Otherwise, the method will be called with one record at a time. The size of the batch is controlled by Kafka consumer properties max.poll.records, fetch.min.bytes, fetch.max.wait.ms; refer to the Kafka documentation for more information.

Bear in mind that batch mode is not supported with @StreamListener - it only works with the newer functional programming model. reference: https://docs.spring.io/spring-cloud-stream-binder-kafka/docs/current/reference/html/spring-cloud-stream-binder-kafka.html#pause-resume

Spring Cloud Stream - functional and reactive

• Spring Cloud Function
• Cloud Events and Spring - part 1 (Oleg Zhurakousky)
• Cloud Events and Spring - part 2 (Oleg Zhurakousky)
• Spring Cloud Stream - demystified and simplified
• Spring Cloud Stream - functional and reactive (Oleg Zhurakousky)

Spring Cloud Stream Kafka (Part3) Functional Programming

Order Example

Key features:

• spring boot (2.5.13)
• Spring cloud Stream kafka (2020.0.5)
• StageManger
• Functional interface

Documentation (Part 3) Functional Programming

Project Source Code

Spring Cloud Stream Kafka (Part4) Kubernetes & Microservices

Project Source Code

Testing Spring Cloud Stream Applications

Testing Spring Cloud Stream Applications - Part 2 Project Source Code

Relevant articles

• Intro to Apache Kafka with Spring
• Testing Kafka and Spring Boot
• Monitor the Consumer Lag in Apache Kafka
• Send Large Messages With Kafka
• Configuring Kafka SSL Using Spring Boot
• Kafka Streams With Spring Boot
• Baeldung's Relevant Articles Source Code

Official Sample

• Official Sample Source Code

Function Based Stream Application Samples

• Thumbnail Demo

Processing of record batches

• kafka-batch-sample

Spring Cloud Stream Kafka and native encoding

In this Spring Cloud Stream sample, we demonstrate native encoding with Kafka and functions.Spring Cloud Stream will skip the regular message conversion and on the outbound and let Kafka natively perform serialization.

https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/kafka-native-serialization

kafka-streams-samples

https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/kafka-streams-samples

Tools

https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/tools/kafka/docker-compose

Spring Cloud Stream Testing with Embedded Kafka Broker Sample

https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/testing-samples/test-embedded-kafka

How to test Spring Cloud Stream applications ?

This project contains a set of tests for simple Spring Cloud Stream applications to demonstrate what, how and when we can use to test this kind of microservices.
https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/testing-samples/testing-demo

Spring Cloud Stream Sample Transactional Application

https://github.com/spring-cloud/spring-cloud-stream-samples/tree/main/transaction-kafka-samples