Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Status code of URL list downloads isn't checked #510

Open
JustAnotherArchivist opened this issue Jun 17, 2021 · 0 comments
Open

Status code of URL list downloads isn't checked #510

JustAnotherArchivist opened this issue Jun 17, 2021 · 0 comments
Labels
bug pipeline

Comments

@JustAnotherArchivist
Copy link
Contributor

DownloadUrlFile does not verify that the server responded with an HTTP 200. This morning, there was an issue, which lead to lots of errors and occasional 502s. The latter were saved to the -urls.txt file, which obviously then led to further errors. Should probably use r.raise_for_status to take care of that.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug pipeline
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JustAnotherArchivist