Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recoverylog: track multiple segments per file #236

Open
jgraettinger opened this issue Nov 18, 2019 · 0 comments
Open

recoverylog: track multiple segments per file #236

jgraettinger opened this issue Nov 18, 2019 · 0 comments

Comments

@jgraettinger
Copy link
Contributor

As an operator, I'd like recovery logs to track multiple segments of written journal range for every file, up to a bound. For RocksDB workloads, all writes of SST files co-occur closely in the log, but writes to the MANIFEST may be interspersed with many other files. If the FSM machinery tracked disjoint segments of the most recent MANIFEST, it allows playback to skip through portions of the log it's unable to skip today, improving recovery times.

The FSM cannot track every segment of every file, so there must be an upper bound, and the FSM must intelligently collapse Segments to not violate that bound (eg, by picking Segments to merge which least-impact readers).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Gazette Core
  
Inbox
Development

No branches or pull requests

1 participant