Newscast Explorer

---

---

Technologies used

Please check the TechStack.md file for details.

Introduction

In this project are going to explore a known EIP known as Saga.

A saga is in its essence, a way to describe a sequence of processes that should occur sequentially in a transactional way.

There are two major Saga types: A choreography and an orchestration Saga types.

A choreography Saga is based on event sourcing. We have multiple decoupled processes that wait for certain events in order to be triggered.

An orchestration Saga is also based on event sourcing, but instead of having the events scattered, it provides a series of rules on how processes should react to certain sub-process results. An orchestration Saga has provisions for rollback processes for example. As soon as an error occurred, all provisioned methods to perform a rollback will be triggered from the point of failure. This follows the direct opposite direction of the successful route. In this saga, we lay out a chain of processes and for each process we can optionally define a rollback process or any other process we want to trigger should a fail occur.

Sagas work in a different way than chained processes in the sense that they are thought out to be design to predict failures and act upon them. Whether we choose Choreography or Orchestration, we always get an out-of-the-box plan to go forward and go back.

Data flow steps

---

In this project we have two steps:

Fetching data

We run a process to fetch data from a mock feeder. This is not directly related to Sagas but it is part of the example of the use case. The data is sent in a message node containing an author node. The fetch process will run by default configured take maximum 30 seconds per run and a maximum of 100 messages under that run.

Sending comments

We will send comments via POST requests. If the id's of the page, author and message match, then the sagas will run normally. If the any of the id's fail, the process will perform a reactive chain of processes which will mark these comments as not available in the database. It will perform this in this order in reverse. So this means that if the id of a page does not match, there will be a comment of that page in the database as a dangling reference. It will be marked as not available. The comment for the author and the message will be ignored because the sagas will not allow the chain to continue. If the author id does not match, both page and author comments will be marked as not available. There will be no record of message.

---

Stable releases

---

• 1.0.0 - e986fadced9dd48a0ce2711764097f18927ff0ba

---

Project Layout

• news-cast-explorer-common - Java libraries to support Page, Authors and Messages
• news-cast-explorer-saga-common - Kotlin library to support the comments for the previously mentioned entitites
• news-cast-mock - News feeder simulation. This is where the data starts - Spring - Kotlin module - Port 8081

http://localhost:8081/api/newscast/messages

• news-cast-explorer-fetcher - Makes different runs, scheduled or non schedule. Each run can fetch up to 100 messages in a maximum period of 30 seconds by default - Spring Java 17 module - Port 8080

http://localhost:8080/api/newscast/fetcher/pages

• news-cast-explorer-cdc - A CDC mock service to support message exchange in the Kafka streams - Port 8085
• news-cast-explorer-saga-choreography - A choreography implementation of the Saga Architecture - Spring - Kotlin module - Port 8083

How to run

Request examples with cUrl

---

Example request:

curl -X POST http://localhost:8083/api/saga/choreography -H 'Content-Type: application/json' --data '{ "idPage": 1, "pageComment": "I love this", "idAuthor": 2, "authorComment": "This is my favourite author", "idMessage": 3, "messageComment": "I agree", "authorRequestId":123,"pageRequestId":456,"messageRequestId":789 }'

• news-cast-explorer-saga-orchestration - An orchestration implementation of the Saga Architecture - Spring - Kotlin module - Port 8082

Example request:

curl -X POST http://localhost:8082/api/saga/orchestration -H 'Content-Type: application/json' --data '{ "idPage": 1, "pageComment": "I love this", "idAuthor": 2, "authorComment": "This is my favourite author", "idMessage": 3, "messageComment": "I agree", "authorRequestId":123,"pageRequestId":456,"messageRequestId":789 }'

---

GUI Endpoints

• news-cast-explorer-fe - A front end tool providing and interface to visualize the results of the news feed fetcher

http://localhost:9000/

Swagger UI Endpoints

---

• Local run

1. news-cast-explorer-fetcher
2. news-cast-explorer-saga-choreography
3. news-cast-explorer-saga-orchestration
4. news-cast-mock

• Local Docker run

1. news-cast-explorer-fetcher
2. news-cast-explorer-saga-choreography
3. news-cast-explorer-saga-orchestration
4. news-cast-mock

---

How to run

---

make dcup-full-action

---

How to run Cypress

---

Against Nginx

make cypress-open-docker

Directly against the service ports

make cypress-open

---

Java version

---

sdk install java 17-open
sdk use java 17-open

---

Local hosts configuration

---

Running the containers may require you to add this line to your /etc/hosts file:

127.0.0.1       news-cast-kafka

This is mostly because of the way kafka operates in giving responses back

---

Roadmap

Roadmap

---

For roadmap information and current developments please check document Roadmap.md

---

References

• Eventuate.IO
• Saga: How to implement complex business transactions without two phase commit
• Managing data consistency in a microservice architecture using Sagas - Implementing an orchestration-based saga
• Managing data consistency in a microservice architecture using Sagas - Implementing a choreography-based saga
• Choreography pattern with Springboot
• Spock Framework Reference Documentation
• Interaction Based Testing with Spock
• Bash tips: Colors and formatting (ANSI/VT100 Control sequences)
• JUnit 5 Parameter Resolution Example

About me