Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider regularly resetting factset stable/volatile balances #3956

Open
rbrw opened this issue Apr 4, 2024 · 1 comment
Open

Consider regularly resetting factset stable/volatile balances #3956

rbrw opened this issue Apr 4, 2024 · 1 comment
Labels

Comments

@rbrw
Copy link
Contributor

rbrw commented Apr 4, 2024

In an attempt to decrease the steady-state Postgres write load PuppetDB currently stores each factset in two separate jsonb fragments, stable and volatile. Whenever a fact changes, the top-level subtree that contains it is moved from stable to volatile, if it wasn't already in volatile, and never moves it back. So if a leaf fact in a large subtree changes once, and then never changes again, the entire subtree containing that fact will always add to the write load when updating that factset, something the split is intended to avoid. (See the footnote in #3955 for additional information.)

To avoid that, and as a fairly simple improvement, consider moving all of a factset's volatile facts back into stable on some cadence. For example, perhaps make the move during any given factset update based on a percentage that would reset a fact every day/week/... assuming the default factset submission rate,

Thanks to Austin Blatt for the initial suggestion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant