(redo)ticdc: fix the event orderliness in redo log #11117
Conversation
ti-chi-bot
bot
added
release-note-none
ti-chi-bot
bot
added
size/XL
Codecov ReportAttention: Patch coverage is
Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more. @@ Coverage Diff @@
## master #11117 +/- ##
================================================
+ Coverage 57.5466% 57.6426% +0.0959%
================================================
Files 853 854 +1
Lines 125679 126266 +587
================================================
+ Hits 72324 72783 +459
- Misses 47986 48102 +116
- Partials 5369 5381 +12 |
ti-chi-bot
bot
added
size/XXL
cdc/redo/reader/reader.go
Outdated
| h[i].data.RedoRow.Row.StartTs < h[j].data.RedoRow.Row.StartTs { | ||
| return true | ||
| if h[i].data.RedoRow.Row.CommitTs == h[j].data.RedoRow.Row.CommitTs { | ||
| if h[i].data.RedoRow.Row.StartTs < h[j].data.RedoRow.Row.StartTs { |
There was a problem hiding this comment.
remove useless else branch:
| if h[i].data.RedoRow.Row.StartTs < h[j].data.RedoRow.Row.StartTs { | |
| if h[i].data.RedoRow.Row.StartTs != h[j].data.RedoRow.Row.StartTs { | |
| return h[i].data.RedoRow.Row.StartTs < h[j].data.RedoRow.Row.StartTs | |
| } | |
| ... |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: CharlesCheung96, lidezhu The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
|
/retest |
1 similar comment
|
/retest |
|
/retest |
1 similar comment
|
/retest |
|
/cherry-pick release-8.1 |
|
@hongyunyan: once the present PR merges, I will cherry-pick it on top of release-8.1 in the new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
/test cdc-integration-kafka-test |
| return h[i].data.RedoRow.Row.StartTs < h[j].data.RedoRow.Row.StartTs | ||
| } | ||
| // in the same txn, we need to sort by delete/update/insert order | ||
| if h[i].data.RedoRow.Row.ToRowChangedEvent().IsDelete() { |
There was a problem hiding this comment.
ToRowChangedEvent should only be called once, if it may cost some resource.
else is redundant I think.
| expect bool | ||
| }{ | ||
| { | ||
| name: "Delete before Update", |
There was a problem hiding this comment.
These data is hard to read, use schema test helper to new events is preferred.
I would suggest that test for each case independently, instead of create a tests and loop over each.
ti-chi-bot
added
the
needs-cherry-pick-release-8.1
|
@hongyunyan: new pull request created to branch In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/tiflow#release-8.1 from head ti-chi-bot:cherry-pick-11117-to-release-8.1: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-11117-to-release-8.1."}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request"} |
lidezhu
added
the
needs-cherry-pick-release-7.5
|
/run-cherry-picker |
|
In response to a cherrypick label: new pull request created to branch |
lidezhu
added
needs-cherry-pick-release-6.5
|
/run-cherry-picker |
|
In response to a cherrypick label: new pull request created to branch |
|
In response to a cherrypick label: new pull request created to branch |
What problem does this PR solve?
Issue Number: close #11096
What is changed and how it works?
Check List
Tests
Unit test
Manual test (add detailed scripts or steps below)
create mysql sink changefeed with redo log on
run gotpc workload
After 30m, pause changefeed and run redo apply
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note