Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Walsender streaming - log time taken in decoding, reorder buffer during streaming #22143

Open
yugabyte-ci opened this issue Apr 25, 2024 · 0 comments
Open

Walsender streaming - log time taken in decoding, reorder buffer during streaming #22143

yugabyte-ci opened this issue Apr 25, 2024 · 0 comments
Assignees
@dr0pdb
Labels
area/cdc Change Data Capture area/cdcsdk CDC SDK jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@yugabyte-ci
Copy link
Contributor

yugabyte-ci commented Apr 25, 2024 •
edited by jira bot

Jira Link: DB-11071
@yugabyte-ci yugabyte-ci added area/cdc Change Data Capture area/cdcsdk CDC SDK jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Apr 25, 2024
dr0pdb added a commit that referenced this issue Apr 25, 2024
@dr0pdb
[#22143] YSQL: Log the time taken in yb_decode and reorder buffer
5e24eff 
Summary:
This revision adds computation and logging of the time taken by the Walsender in yb_decode and reorderbuffer while processing a single batch from the CDC
service.
Jira: DB-11071

Test Plan:
Jenkins: compile only

Existing tests. Looked at the log manually in a local test run.

Reviewers: asrinivasan

Reviewed By: asrinivasan

Subscribers: ycdcxcluster, yql

Differential Revision: https://phorge.dev.yugabyte.com/D34530
dr0pdb added a commit that referenced this issue Apr 29, 2024
@dr0pdb
[BACKPORT 2024.1][#21950][#22133][#22143] YSQL: Backport Walsender de…
b35c588 
…bug logs and unacked txn fix

Summary:
##### Backport Description
All merges were clean. No conflicts.

##### Original Description
Original commits:
1b902ba / D34327
48a0279 / D34483
5e24eff / D34530

###### YSQL: Insert transaction into unacked txn list only upon receipt of commit record
We maintain a list of unacked transactions for the calculation of the restart_lsn from the confirmed_flush lsn. Prior to this revision, this list also included a transaction at
the end for which we haven't yet received the COMMIT record from the CDC service. Such a transaction is stored with commit_lsn as InvalidXLogRecPtr (0) and was leading to issues in the calculation in the restart_lsn.

This revision updates the logic to put a transaction into the list only on the receipt of the commit record.
Jira: DB-10866

###### YSQL: Log the time taken in converting from QLValuePB to PG datum in walsender
The walsender spends a considerable amount of time in converting QLValuePB to PG datum in ybc_pggate. Add a VLOG(1) which logs the time taken in this
operation.
Jira: DB-11059

###### YSQL: Log the time taken in yb_decode and reorder buffer
This revision adds computation and logging of the time taken by the Walsender in yb_decode and reorderbuffer while processing a single batch from the CDC
service.
Jira: DB-11071

Test Plan: Jenkins: test regex: .*ReplicationSlot.*

Reviewers: asrinivasan, skumar

Reviewed By: skumar

Subscribers: yql, ycdcxcluster

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D34535
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dr0pdb dr0pdb

Labels
area/cdc Change Data Capture area/cdcsdk CDC SDK jira-originated kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yugabyte-ci @dr0pdb