Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support stale-while-revalidate items (swrv) #94

Open
thibaultcha opened this issue Jun 23, 2020 · 0 comments
Open

Support stale-while-revalidate items (swrv) #94

thibaultcha opened this issue Jun 23, 2020 · 0 comments
Labels

Comments

@thibaultcha
Copy link
Owner

thibaultcha commented Jun 23, 2020

It can be extremely useful for mlcache to return stale items while the underlying L3 callback is being run by a worker. Unlike resurrect_ttl, which resurrects an item and marks it as stale when the L3 callback returns nil, err (e.g. database lookup error), support for swrv items would mean that an item is marked as stale by mlcache (and callers of get() can be made aware of the stale state of the fetched item), all the while an L3 callback is being run in the background by a given worker.

Currently, implementing swrv implicates scheduling background timers to fetch fresh data. Once fetched, users call cache:set() to update the data in the L2 cache and each worker's L1 cache. This forces users to call cache:update() as well to ensure that all workers L1 is up-to-date.

We should try to implement swrv items natively, without having to rely on any IPC. Items could be marked as stale and be scheduled for re-validation in a number of ways:

  1. Via a new stale_ttl option: when reached, the item is not removed from the cache like ttl, but instead is marked as stale (for the cache:get() hit_lvl return value) and its associated L3 callback is scheduled in a timer. This approach requires that the callback argument be stored inside of mlcache for later use when stale_ttl is reached.
  2. When returning from the L3 callback, a 4th return value could be: item, err, ttl, stale. When stale is truthy, the item is marked as stale and only stored in L2 (never promoted to L1). Subsequent calls to get() return the L2 data and an appropriate hit_lvl indicating the staleness. Eventually, the fresh data is fetched by the user (outside of mlcache) who calls cache:set_fresh(), which only updates the L2 data and removes the stale marker, causing the data to be promoted to L1 again.

Each approach comes with its own pros and cons or opportunities for new APIs (e.g. custom swrv and retry strategies implemented in an OOP fashion) which I won't detail right now...

Let's talk about implementation here before working on any PR for this!

@thibaultcha thibaultcha changed the title [enhancement] support stale-while-revalidate items (swrv) [feature] support stale-while-revalidate items (swrv) Sep 25, 2020
@thibaultcha thibaultcha changed the title [feature] support stale-while-revalidate items (swrv) Support stale-while-revalidate items (swrv) Aug 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant