Caching query results

[dumbbell]

Currently, all queries go through the Ra server process because we need the state machine's state to fufill a query. This could be expensive in a workload with a lot of queries. We could cache the result of queries to improve this situation.

I'm dropping some random ideas below:

• We could use an ETS table.
• Updates would invalidate cache entries.
• We will need a cache replacement policy.
• We may need some kind of memory monitoring to shrink/empty the cache if necessary.

[dumbbell]

@dcorbacho started to work on caching in the khepri-machine-cache branch. The prototype is based on an ETS table. The table is filled and updated by khepri_machine, the Ra state machine. The cache is queried first when servicing queries before a Ra server is contacted. Results are promising, but there is still work to do.

In particular, here are some issues or missing features:

• The Ra state machine code should not have any side effects, but it does now. Therefore the update of the cache should be moved to a proper side effect outside of the state machine.
• There is no cache replacement policy and anything to limit the memory footprint of the cache.

Another idea I had this morning: perhaps we could cache paths selected by the caller only. Like, the caller could perform the following write:

khepri_machine:put(StoreId, Path, Payload, #{cache => true}).

Other paths would not be stored in the cache. Once we have memory management in place, having that would help memory management and increase the chance that wanted paths are present in the cache. Paths that the caller doesn't need in the cache will thus never cause wanted paths to be cleared from that cache.

We could provide a store-level option to cache everything.

[dumbbell]

Another way to implement path-specific caching would to leverage what we have in triggers: if a path matches a path pattern, cache it.

[the-mikedavis]

Should only the Ra leader cache query results?

I think it would be a nice speed boost for queries with a low_latency favor to read from a local cache. In order for that to work we would need to allow all Ra cluster members to each affect their own ETS tables which conflicts with how Ra treats effects.

Edit: we can use Ra's auxiliary commands to propagate changes to the cache to all replicas, so a cache local to each server in the cluster should be possible.

[leeyisoft]

I suggest you refer to this：
depcache is an in-memory caching server for Erlang with dependency checks, cache expiration and local in process memoization of lookups. https://github.com/zotonic/depcache

[the-mikedavis]

Depcache has some features that would probably not be useful in this scenario like time-to-live and cache dependencies. Ideally the query cache should be a behaviour and cache implementations should be pluggable though, so it may be possible to tell Khepri to use depcache as the cache backend.