Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write the "Storage Engine" RFD #13

Open
alilleybrinker opened this issue Apr 1, 2024 · 0 comments
Open

Write the "Storage Engine" RFD #13

alilleybrinker opened this issue Apr 1, 2024 · 0 comments
Labels
status: needs-rfd This topic is big enough to require an RFD. type: enhancement New feature or request
Milestone

Comments

@alilleybrinker
Copy link
Collaborator

One of the challenges with Hipcheck is that the scale of data needing to be analyzed can quickly outgrow the capacity of memory, and ideally we want to manage this ourselves to begin backing this down to disk. There's extensive literature on how to handle this kind of caching, and Hipcheck does present one benefit over other theoretical systems, in that we know up front what data is expected to be collected by the workload being run, and can try to make intelligent decisions about how that data will be stored.

Related to the API design, we'll need to provide some mechanism for data to be sent or shared between plugins and the central Hipcheck engine, and we'll have to determine the level of control plugins will be given over that storage and transmission.

The storage engine RFD should reflect how data is stored, what controls are available to Hipcheck users and to plugins to impact that storage, and ideally how storage can be introspected and debugged.

@alilleybrinker alilleybrinker added type: enhancement New feature or request status: needs-rfd This topic is big enough to require an RFD. labels Apr 1, 2024
@alilleybrinker alilleybrinker added this to the 3.3.0 milestone Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: needs-rfd This topic is big enough to require an RFD. type: enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

1 participant