Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor summaries computation to allow for different time intervals #524

Open
muety opened this issue Sep 21, 2023 · 1 comment
Open

Refactor summaries computation to allow for different time intervals #524

muety opened this issue Sep 21, 2023 · 1 comment

Comments

@muety
Copy link
Owner

muety commented Sep 21, 2023

Currently, summaries are persisted on a per-day level. Moreover, the summary calculation logic assumes that there are no overlaps.

For performance reasons, it would be cool to have support for storing summaries on different levels of granularity, e.g. have daily summaries alongside monthly or yearly summaries.

Then, for when a request comes in, there are three options how we could make Retrieve() behave:

  • Option A: Make it as smart as to "adaptively" choose from the available summaries while (a) avoiding overlaps and (b) minimize minimizing the number of individual summaries used
  • Option B: Hard-code what "granularity levels" could theoretically exist, then start with the largest one that still matches the requested interval, probe to find according persisted summaries and otherwise continue with the next smaller one
  • Option C: Explicitly store the "granularity" level with a summary (e.g. have an interval column with values daily, weekly, etc., similar to what we have for leaderboard_items) and then follow the logic from option B of choosing the appropriate granularity level given the requested interval

I think option C is easiest to implement. However, this needs to be planned out further.

@muety
Copy link
Owner Author

muety commented Sep 22, 2023

Also, allow to persist filtered summaries (see below) to speed up a bunch of different queries.

// Filtered summaries are not persisted currently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant