Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to create temporary files for partial downloads #122

Open
dreamflasher opened this issue Jan 11, 2023 · 5 comments
Open

Add option to create temporary files for partial downloads #122

dreamflasher opened this issue Jan 11, 2023 · 5 comments
Labels
enhancement A new feature or other enhancement to the library good first issue help wanted

Comments

@dreamflasher
Copy link
Contributor

Currently, if the process is killed it's impossible to find out if a file was completely downloaded. Thus it would be great to add an option that files are created with a prefix or suffix and renamed to the actual filename only if they are completed.

@Cadair
Copy link
Owner

Cadair commented Mar 28, 2023

If the download is interrupted (but not completely killed) the partially downloaded files should be removed.

I agree that renaming at the end of the download is a good idea, shouldn't be that hard to add.

@Cadair Cadair added enhancement A new feature or other enhancement to the library help wanted good first issue labels Mar 28, 2023
@dreamflasher
Copy link
Contributor Author

I also had situations where an error occured, but the partially downloaded file wasn't deleted. My current impression is that the results object is not containing all errors, but I need to do proper debugging (it's a bit challening, because it occured when I downloaded ~1000 files, so there's a bunch of randomness involved).

What do you think, should renaming at the end of the download be the default? (Instead of the callback handler? With the callback handler I was inspired by https://docs.python.org/3/library/asyncio-task.html#asyncio.Task.add_done_callback – but maybe it adds too much complexity here)

@Cadair
Copy link
Owner

Cadair commented May 9, 2023

should renaming at the end of the download be the default

yes, I think so.

Instead of the callback handler?

I think from a user perspective they are separate, but we could use the callback handler internally if that makes sense.

@dreamflasher
Copy link
Contributor Author

Great. Yes, I think the callback handler would be a great usecase for implementing this. Lets finish that PR and then I'll add the temp-file callback handler. Or do you prefer that I add the temp-file callback handler already in the other PR?

@Cadair
Copy link
Owner

Cadair commented May 16, 2023

Keeping them separate is a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or other enhancement to the library good first issue help wanted
Projects
None yet
Development

No branches or pull requests

2 participants