Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byte ranges need to permit reading from the end #266

Open
LDeakin opened this issue Sep 26, 2023 · 0 comments
Open

Byte ranges need to permit reading from the end #266

LDeakin opened this issue Sep 26, 2023 · 0 comments

Comments

@LDeakin
Copy link

LDeakin commented Sep 26, 2023

The specification mentions that the decoded_regions parameter in c.partial_decode() specifies a list of byte ranges. With the sharding codec, to read the shard index without reading the whole file it must be possible to read some number of bytes from the end of the input_handle.

In the Abstract Store Interface section, a byte range is specified as

To read and write partial values, a range specifies two integers range_start and range_length, that specify a part of the value starting at byte range_start (inclusive) and having a length of range_length bytes. range_length may be none, indicating all available data until the end of the referenced value.

I think that paragraph should be updated to reflect that a byte range can be

  • relative to the start of the value (supporting reading to the end), or
  • relative to the end of the value (supporting reading to the start).

Additionally, I think the details of how a byte range should be implemented (e.g. enum or negative offset/size) should be left up to the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant