Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data Versioning #283

Open
MosGeo opened this issue Dec 28, 2023 · 1 comment
Open

Data Versioning #283

MosGeo opened this issue Dec 28, 2023 · 1 comment

Comments

@MosGeo
Copy link

MosGeo commented Dec 28, 2023

This is early thoughts but has anybody thought about the possiblity of including some kind of data versioning specs in Zarr. The use would be that multiple versions of the same data can be stored and Zarr would only keep multiple copies of the changed chunks. This would save a lot of space if the data just changed a little bit.

from practical perspective, Zarr would return the latest version the chunk unless version is specified. In that case, it woulo return the latest version up to the specified version.
@jhamman
Copy link
Member

jhamman commented Dec 28, 2023

Hi @MosGeo - there is a relevant discussion in #154.

tldr; there are a few efforts in this space including our product Arraylake (versioning docs) and Tensorstore/OctDB (docs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jhamman @MosGeo