Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #4852] Support Jraft algorithm as meta storage #4862

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

karsonto
Copy link
Contributor

@karsonto karsonto commented Apr 24, 2024

Fixes #4852

Motivation

Support Jraft algorithm as meta storage

Modifications
Removed unused variable responseEventMeshCommand in SendSyncMessageProcessor#processRequest

Documentation
New Configuration Information Explanation

  • eventMesh.metaStorage.raft.dataPath=/tmp/server1 // Path to local snapshot storage for raft data
  • eventMesh.metaStorage.raft.self=127.0.0.1:9091 // Raft protocol local ip and port
  • eventMesh.metaStorage.raft.members=127.0.0.1:9092,127.0.0.1:9093 // Raft members node ip and port
  • eventMesh.metaStorage.raft.electionTimeout=5 // Raft Election Leader Timeout second
  • eventMesh.metaStorage.raft.snapshotInterval=30 // Raft snapshot Interval Timeout second
  • eventMesh.metaStorage.raft.refreshLeaderInterval=3 // Raft refresh Leader node Interval Timeout second

@Pil0tXia
Copy link
Member

Pil0tXia commented Apr 24, 2024

Please be noted that you're linking an irrelavant issue.

@karsonto karsonto changed the title [ISSUE #4825] Support Jraft algorithm as meta storage [ISSUE #4852] Support Jraft algorithm as meta storage Apr 24, 2024
@Pil0tXia
Copy link
Member

May you please introduce briefly that how JRaft stores metadata and how to retrieve them (such as subscription info) from clients (such as EventMesh Dashboard or user)?

Copy link
Member

@Pil0tXia Pil0tXia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall good job.

eventmesh-meta/eventmesh-meta-raft/build.gradle Outdated Show resolved Hide resolved
eventmesh-meta/eventmesh-meta-raft/build.gradle Outdated Show resolved Hide resolved
Comment on lines 88 to 95
# metaStorage plugin: raft
#eventMesh.metaStorage.raft.dataPath=/tmp/server1
#eventMesh.metaStorage.raft.self=127.0.0.1:9091
#eventMesh.metaStorage.raft.members=127.0.0.1:9092,127.0.0.1:9093
#eventMesh.metaStorage.raft.electionTimeout=5
#eventMesh.metaStorage.raft.snapshotInterval=30
#eventMesh.metaStorage.raft.refreshLeaderInterval=3

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May you please leave a usage document/comment/URL that explains the meaning of these properties?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eventMesh.metaStorage.raft.dataPath=/tmp/server1 // Path to local snapshot storage for raft data

Is server1 the recommended path of JRaft? Maybe /tmp/eventmesh-meta-raft would be better to understand.

eventMesh.metaStorage.raft.members=127.0.0.1:9092,127.0.0.1:9093 // Raft members node ip and port

My guess is that the IP addresses are different for different nodes, but the port should be the same to allow scaling. The example here requires changing the IP address and maintaining the default port number.


May you please add these comments into eventmesh.properties?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this example was configured locally for ease of testing, and it should be more appropriate to keep the same port numbers

@karsonto
Copy link
Contributor Author

May you please introduce briefly that how JRaft stores metadata and how to retrieve them (such as subscription info) from clients (such as EventMesh Dashboard or user)?

My understanding, Jraft is to synchronise the data to the cluster internal machine, each machine above the data is consistent, this PR is to save the data in memory, every once in a while will snapshot the data to the local disk, start the server will snapshot the data loaded into memory, EventMesh Runtime needs to provide restful api in order to allow EventMesh Dashboard to get the data saved by the jraft!

@Pil0tXia
Copy link
Member

EventMesh Runtime needs to provide restful api in order to allow EventMesh Dashboard to get the data saved by the jraft!

May you please indicate which class to interact with in order to get JRaft data?

@karsonto
Copy link
Contributor Author

EventMesh Runtime needs to provide restful api in order to allow EventMesh Dashboard to get the data saved by the jraft!

May you please indicate which class to interact with in order to get JRaft data?

You can get MetaStorage in the Runtime, and then get MetaService, which is the implementation class of RaftMetaService.

@Pil0tXia
Copy link
Member

Please add known dependencies~

@Pil0tXia
Copy link
Member

Implemented JRaft Feature in this PR

In the current implementation of this PR, if a new node is added to the cluster, users need to add the new node to the eventmesh.properties of the existing nodes before it can be added to the cluster.

Hopefully you can implement CliService in a next PR to support service registration when automatic scaling up and down of the cluster. Because the complete list of online nodes of the cluster cannot be maintained in real time.

After it's implemented, the newly added node can connect to any online node in the cluster. If connected to a follower, the current leader will be returned to new node by the follower.

Copy link
Member

@Pil0tXia Pil0tXia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@karsonto
Copy link
Contributor Author

If you get bugs during use, I will continue to improve it later, thanks!

@Pil0tXia Pil0tXia added the ready for review PR is waiting for reviewer's approval or opinion (used as a strong reminder) label Apr 30, 2024
Copy link
Member

@mxsm mxsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@karsonto Please fix conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review PR is waiting for reviewer's approval or opinion (used as a strong reminder)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Support Jraft algorithm as meta storage
3 participants