New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(pruning): fair pruning #8180
base: main
Are you sure you want to change the base?
Conversation
still have to build some tests |
It feels a bit overcomplicated, can we just have a How I see it:
WDYT? |
I am not sure if we need it, what is the case when we want to continue pruning some table inside a segment, and not the whole segment from the beginning? |
No need to reallocate memory, easiest is to just save the index we would have pruned next in the True that there is no need to generate the segments, other than for static files, on each call to |
checkpoints are saved when pruning stops |
up to you, I'd prefer a "Since VecDeque is a ring buffer, its elements are not necessarily contiguous in memory." – from https://doc.rust-lang.org/std/collections/struct.VecDeque.html |
don't think it makes that big difference now that this is implemented + tested |
Closes #7343, related to pruner interruption ref #6770.