WIP: Retries #232
Draft
WIP: Retries #232
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Still need to do sleep/backoff, etc. Hoping to remove NodataReader too.
When an `open` fails and we'd return nodata instead, we don't cache that anymore. So if the asset doesn't exist, we'll try to open it for every chunk. Unclear how much of a performance impact this will have. It's probably not ideal, because combining it with retries could be nice: retry errors first, then give up and use nodata if they persist?
got lost in #221, which was a reasonable change, but turns out there was still a test importing it
TODO: - sleep & backoff - figure out how to handle sleep in tests
|
Hi @gjoseph92 thanks for this feature. I had a look an although I have not tested the code it looks good. The way of using this feature seems versatile so imho, nice, I would defiantly use this feature! I see that you have not included the Also, I would think that the possibility to add a delay between retries would be also useful. |
|
Hello @gjoseph92 . Do you think you will merge this anytime soon? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support retrying errors matching a given pattern.
This also moves error handling logic (both retires and nodata) up to the
to_dasklevel, and out of Readers.Closes #18.