Raftstore: async-io brings more minor IOs when writing raft logs. #16907
Assignees
Labels
Comments
9 tasks
|
/cc @Yui-Song The previous regression issue related summary and improvement is followed in this issue. |
ti-chi-bot
pushed a commit
to ti-chi-bot/tikv
that referenced
this issue
May 9, 2024
ref tikv#16907 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
Merged
9 tasks
ti-chi-bot bot
pushed a commit
that referenced
this issue
May 9, 2024
…ics on it. (#16906) (#16967) ref #16907 This pr contains: - Reduce the default compression threshold in RaftEngine to `4kb` as default if users do not manually set it. - Add relevant metrics on the compression ratio of RaftEngine.write. Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: lucasliang <nkcs_lykx@hotmail.com>
|
@LykxSassinator Are there any pending tasks for this issue? Shall we close this one? |
|
Yep, as #16967 has been merged to v8.1, this issue can be closed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Development Task
Previous work in #16615 shows that the performance on write workloads can be improved as expected by enabling
async-ioas default.However, we find that
async-iowill make side effects to the log batching mechanism, which will make each batch of writing smaller than disablingasync-io, according to the metrics from RaftEngine. And smaller batch will make it cannot reach the threshold of compression. Then, it will increase the costs on IO bandwidth (nearly ~10%).async-ioasync-ioAnd when we increase the threads of
async-io, the smaller batching gives higher costs on IOPS.Finally, by reducing the
raft-engine.batch-compression-thresholdto4kbfrom8kb(default), the costs onasync-iocan be eliminated as expected.So, we should give a more reasonable setting on the compression threshold after enabling
async-io.The text was updated successfully, but these errors were encountered: