Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tiered compaction: switch to vectored get #7659

Closed
Tracked by #7554
problame opened this issue May 8, 2024 · 0 comments · Fixed by #7850
Closed
Tracked by #7554

tiered compaction: switch to vectored get #7659

problame opened this issue May 8, 2024 · 0 comments · Fixed by #7850
Assignees
@arpad-m
Labels
c/storage/pageserver Component: storage: pageserver

Comments

@problame
Copy link
Contributor

problame commented May 8, 2024 •
edited

Tiered compaction was written & integrated concurrently to the integration of vectored get.

So, tiered compaction doesn't use vectored get, but single Timeline::get calls.

Using vectored get would be more efficient.

Even more efficient would be to have a special access mode for compaction.

But, one step at a time.
@problame problame mentioned this issue May 8, 2024
Open
@problame problame added the c/storage/pageserver Component: storage: pageserver label May 8, 2024
@arpad-m arpad-m mentioned this issue May 22, 2024
Merged
arpad-m added a commit that referenced this issue May 23, 2024
@arpad-m @skyzh
Use Timeline::create_image_layer_for_rel_blocks in tiered compaction (#…
75a52ac 
…7850)

Reduces duplication between tiered and legacy compaction by using the
`Timeline::create_image_layer_for_rel_blocks` function. This way, we
also use vectored get in tiered compaction, so the change has two
benefits in one.

fixes #7659

---------

Co-authored-by: Alex Chi Z. <iskyzh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@arpad-m arpad-m

Labels
c/storage/pageserver Component: storage: pageserver
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use Timeline::create_image_layer_for_rel_blocks in tiered compaction neondatabase/neon
2 participants
@problame @arpad-m