Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

progress=False and overwrite=True are ignored by simple_download() #114

Open
ebuchlin opened this issue Nov 2, 2022 · 1 comment
Open

Comments

@ebuchlin
Copy link

ebuchlin commented Nov 2, 2022

With the following code

from parfive import Downloader
files = [
    'https://idoc-regards-data.ias.u-psud.fr/SDO_DEM/2012/08/DEM_aia_2012-08-10T00_05.tar',
    'https://idoc-regards-data.ias.u-psud.fr/SDO_DEM/2012/08/DEM_aia_2012-08-10T01_05.tar',
    'https://idoc-regards-data.ias.u-psud.fr/SDO_DEM/2012/08/DEM_aia_2012-08-10T02_05.tar',
    'https://idoc-regards-data.ias.u-psud.fr/SDO_DEM/2012/08/DEM_aia_2012-08-10T03_05.tar',
]
dl = Downloader(progress=False, overwrite=True)
print('With enqueue_file() and download():')
for f in files: dl.enqueue_file(f, path='result')
dl.download()
print('Done')
print('With simple_download():')
dl = Downloader(progress=False, overwrite=True)
dl.simple_download(files, path='result')
print('Done')

the resulting terminal output is

With enqueue_file() and download():
Done
With simple_download():
Files Downloaded: 100%|███████████████████████████████████████████████| 4/4 [00:00<00:00, 31.12file/s]
Done

Then:

  • enqueue_file() then download() takes progress=False and overwrite=True into account
  • a progress bar is shown at the second download, so progress=False is ignored by simple_download()
  • the download speed indicates that overwrite=True is also ignored by simple_download(); removing the files before the second download results in a much slower (and realistic) download speed
Files Downloaded: 100%|███████████████████████████████████████████████| 4/4 [00:21<00:00,  5.28s/file]

Expected behavior: progress=False and overwrite=True should not be ignored by simple_download()

Specifying overwrite=True in the call to simple_download() works, but the expectation is that overwrite from Downloader is inherited by default when calling simple_download() without specifying overwrite.

@ebuchlin
Copy link
Author

ebuchlin commented Nov 2, 2022

We just understood that this is because simple_download() is a classmethod, but for some reason I was confused about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant