Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify UI backend caching and improve scalability #266

Open
saikonen opened this issue Nov 13, 2021 · 1 comment
Open

Simplify UI backend caching and improve scalability #266

saikonen opened this issue Nov 13, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@saikonen
Copy link
Collaborator

Suggested improvement
Caching solution used by the UI backend could be changed to one that performs simple function caching, and one that uses a cache store that is shareable across multiple instances (for example with Redis). There are some promising async caching solutions like https://github.com/aio-libs/aiocache which these features.

Motivation
Current caching layer implementation for the ui_backend_service is instance specific, making horizontal scaling have unnecessary overhead. There is also a lot of boilerplate associated with computing values, caching them, and accessing the cached values. Function caching would also make multi-layer caches easier to implement compared to the current CacheAction approach.

@saikonen saikonen added the enhancement New feature or request label Nov 13, 2021
@saikonen
Copy link
Collaborator Author

The biggest hurdle for a change such as this is the UI services reliance on the Metaflow client. As the client is not async compliant, it would have to be executed in a subprocess and possibly rate-limited for concurrency. Surfacing client errors to the main process would still be desired so these can be conveyed to the UI, so this would require some extra tooling.

The client instances should most likely need to be reusable, as judging from past tests, there is quite a bit of overhead with launching a client instance (and have it grab the required credentials)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant