Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resume partial blob downloads #449

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Resume partial blob downloads #449

wants to merge 1 commit into from

Conversation

Sunjeet
Copy link
Contributor

@Sunjeet Sunjeet commented Jan 31, 2020

This is useful for local dev on spotty internet connection

@Sunjeet Sunjeet requested review from a user, akhaku and dkoszewnik January 31, 2020 01:53
byte buf[] = new byte[4096];
// resume unfinished downloads
File outFile = new File(tempPath.toAbsolutePath().toString());
is.skip(outFile.length());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is better than we have currently, but I'm guessing this results in effectively the same behavior from a stream consumption perspective, i.e. we have to read the bytes (even if we drop them)?

@@ -223,12 +226,17 @@ public InputStream getInputStream() throws IOException {
@Override
public InputStream getInputStream() throws IOException {

Path tempPath = path.resolveSibling(path.getName(path.getNameCount()-1) + "-" + UUID.randomUUID().toString());
Path tempPath = path.resolveSibling(PARTIAL_DOWNLOAD + path.getName(path.getNameCount()-1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, although FWIW most partial download files I've seen end in .part or similar (rather than being prefixed)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is some fragile filename parsing at other places in code, and it seemed to work better to have partial in the prefix as opposed to suffix. One eg. is checking if filename startswith "snapshot.".

HollowConsumer consumer = consumerBuilder.build();
boolean downloadInterrupted = false;
try {
consumer.triggerRefreshTo(testVersion);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor cleanup - you could move the fail call here and not need the downloadInterrupted boolean at all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants