You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Suggested improvement
Caching solution used by the UI backend could be changed to one that performs simple function caching, and one that uses a cache store that is shareable across multiple instances (for example with Redis). There are some promising async caching solutions like https://github.com/aio-libs/aiocache which these features.
Motivation
Current caching layer implementation for the ui_backend_service is instance specific, making horizontal scaling have unnecessary overhead. There is also a lot of boilerplate associated with computing values, caching them, and accessing the cached values. Function caching would also make multi-layer caches easier to implement compared to the current CacheAction approach.
The text was updated successfully, but these errors were encountered:
The biggest hurdle for a change such as this is the UI services reliance on the Metaflow client. As the client is not async compliant, it would have to be executed in a subprocess and possibly rate-limited for concurrency. Surfacing client errors to the main process would still be desired so these can be conveyed to the UI, so this would require some extra tooling.
The client instances should most likely need to be reusable, as judging from past tests, there is quite a bit of overhead with launching a client instance (and have it grab the required credentials)
Suggested improvement
Caching solution used by the UI backend could be changed to one that performs simple function caching, and one that uses a cache store that is shareable across multiple instances (for example with Redis). There are some promising async caching solutions like https://github.com/aio-libs/aiocache which these features.
Motivation
Current caching layer implementation for the ui_backend_service is instance specific, making horizontal scaling have unnecessary overhead. There is also a lot of boilerplate associated with computing values, caching them, and accessing the cached values. Function caching would also make multi-layer caches easier to implement compared to the current CacheAction approach.
The text was updated successfully, but these errors were encountered: