Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db,wal: avoid Batch.refData allocation #3565

Merged
merged 1 commit into from
Apr 30, 2024

Conversation

jbowens
Copy link
Collaborator

@jbowens jbowens commented Apr 29, 2024

Passing in the Batch.refData func into (wal.Writer).WriteRecord forced an allocation for every committed batch. This commit defines a new wal.RefCount interface implemented by Batch to avoid the allocation.

Informs cockroachdb/cockroach#123236.

@jbowens jbowens requested a review from a team as a code owner April 29, 2024 19:34
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @aadityasondhi, @RaduBerinde, and @sumeerbhola)


batch.go line 232 at r1 (raw file):

const batchClosedBit = 1 << 30

// TODO(jackson): Hide the wal.RefCount implementation from the public Batch interface.

Any ideas on how to achieve this?

Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @aadityasondhi, @jbowens, and @RaduBerinde)


batch.go line 232 at r1 (raw file):

Previously, jbowens (Jackson Owens) wrote…

Any ideas on how to achieve this?

Not really. Wrapping Batch in another struct seems unnecessary boilerplate.


wal/wal.go line 389 at r1 (raw file):

	// like to reuse p's memory, the caller may pass a non-nil [RefCount].  If
	// the Writer will retain p, it will invoke the [RefCount] before returning.
	// When it's finished, it will invoke the func returned by the [RefCount] to

nit: func returned by the RefCount is stale.

Passing in the Batch.refData func into (wal.Writer).WriteRecord forced an
allocation for every committed batch. This commit defines a new wal.RefCount
interface implemented by Batch to avoid the allocation.

Informs cockroachdb/cockroach#123236.
Copy link
Collaborator Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @aadityasondhi, @RaduBerinde, and @sumeerbhola)


wal/wal.go line 389 at r1 (raw file):

Previously, sumeerbhola wrote…

nit: func returned by the RefCount is stale.

good catch

@jbowens jbowens merged commit d4ce5b0 into cockroachdb:master Apr 30, 2024
11 checks passed
@jbowens jbowens deleted the refdata-alloc branch April 30, 2024 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants