Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refused connections are not retried #464

Open
JustAnotherArchivist opened this issue Sep 11, 2020 · 1 comment
Open

Refused connections are not retried #464

JustAnotherArchivist opened this issue Sep 11, 2020 · 1 comment
Labels
bug pipeline

Comments

@JustAnotherArchivist
Copy link
Contributor

ArchiveBot does not retry [Errno 111] Connection refused errors. I think it should and consider this a bug. This simply requires adding the wpull option --retry-connrefused.
@JustAnotherArchivist
Copy link
Contributor Author

Real-world example of something that effectively can't be archived due to this without ridiculous manual effort: ShoutWiki. It refuses connections in a pretty much random manner, which then wrecks any recursive crawls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug pipeline
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JustAnotherArchivist