We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I found a bug, not just asking a question, which should be created in GitHub Discussions.
I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.
I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
CentOS 7
develop
OpenJDK 8
目前的分层存储实现没有区分主从节点,所有 broker 会共同向冷存中转储数据,存在数据冲突
以主从模式或 dLedger 模式部署 broker,自行实现对象存储的 FileSegment
分层存储应该正常运行
数据冲突
No response
The text was updated successfully, but these errors were encountered:
当前分级存储对主备模式和 DLedger 的支持不够完善,非常欢迎你把他作为新的特性来贡献。从设计方案上来说,并不只是 schedule 任务判断一下即可,metadata store 中元数据的同步,以及 broker container 模式下的截断等问题都是需要考虑和讨论的,从目前的 pr 来看考虑不够完善。
Sorry, something went wrong.
是的,这个 pr 原本只是打算让分级存储能够在主从/ DLedger 模式下跑起来,后续我计划更新主从同步等部分逻辑
hello,最近在考虑 IndexFile 的主从同步问题,发现如果使用对象存储作为冷存,根据现有逻辑,如果发生切主,由于从节点无法获得未上传的 IndexFile 中消息所指向的冷存中的消费位点,Sealed 与 Unsealed 部分的数据似乎会永久丢失掉,这个问题有什么解决思路吗
目前可以想到的折中方案包括:将 DispatchRequest 攒批同步到从节点,从节点自行构建本地 IndexFile,切为主节点后再做上传;但这个方案问题在于同步过程容易出现 gap,可以预见到切主过程中这段时间的消息将不能通过 IndexFile 查询到
如果要保留的话,一般想法就是等旧主恢复之后做一下 force upload,插入到 map<time, flatFile>
bxfjb
Successfully merging a pull request may close this issue.
Before Creating the Bug Report
I found a bug, not just asking a question, which should be created in GitHub Discussions.
I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.
I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
Runtime platform environment
CentOS 7
RocketMQ version
develop
JDK Version
OpenJDK 8
Describe the Bug
目前的分层存储实现没有区分主从节点,所有 broker 会共同向冷存中转储数据,存在数据冲突
Steps to Reproduce
以主从模式或 dLedger 模式部署 broker,自行实现对象存储的 FileSegment
What Did You Expect to See?
分层存储应该正常运行
What Did You See Instead?
数据冲突
Additional Context
No response
The text was updated successfully, but these errors were encountered: