Add baseline resync feature with read support #387
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hkadayam
reviewed
Apr 18, 2024
| @@ -293,7 +295,13 @@ bool HomeRaftLogStore::compact(ulong compact_lsn) { | |||
| // we rely on resrouce mgr timer to trigger truncate for all log stores in system; | |||
| // this will be friendly for multiple logstore on same logdev; | |||
|
|
|||
| // m_log_store->truncate(to_store_lsn(compact_lsn)); | |||
| #ifdef _PRERELEASE | |||
| if (iomgr_flip::instance()->test_flip("force_home_raft_log_truncate")) { | |||
There was a problem hiding this comment.
Suggest a parameterized flip, so you can force raft log truncate on specific logstore. This will trigger on every logstore truncate always.
hkadayam
reviewed
Apr 18, 2024
| @@ -9,7 +9,7 @@ namespace homestore { | |||
| uint64_t ReplLogStore::append(nuraft::ptr< nuraft::log_entry >& entry) { | |||
| // We don't want to transform anything that is not an app log | |||
| if (entry->get_val_type() != nuraft::log_val_type::app_log) { return HomeRaftLogStore::append(entry); } | |||
|
|
|||
| if (entry->get_buf_ptr()->size() == 0) { return HomeRaftLogStore::append(entry); } | |||
There was a problem hiding this comment.
nit: isn't easier to put || with above condition?
Add nuraft based snapshot sync across leader and followers. Implement the read snapshot callback. Add a generic snapshot structure to support different implemenations. Listener implements necessary function to load/save snapshot state. Listener also implements on how to read and write actual snapshot data. Added test to do basic baseline test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add generic snapshot context and data structure.
Add callbacks for listener as discussed.
Use nuraft read and write logical snapshot callbacks to transfer the data.
Do test on both read and write for the baseline resync.
Key and values were pushed by leader and written by follower and its tested.
Currently verification of the data in the test are disabled but have been locally tested.