Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recover from HTTP errors like read timeout #248

Open
nemobis opened this issue Feb 20, 2020 · 0 comments
Open

Recover from HTTP errors like read timeout #248

nemobis opened this issue Feb 20, 2020 · 0 comments

Comments

@nemobis
Copy link

nemobis commented Feb 20, 2020

  File "/home/users/federico/.local/lib/python2.7/site-packages/mwclient/client.py", line 286, in api
    info = self.raw_api(action, http_method, **kwargs)
  File "/home/users/federico/.local/lib/python2.7/site-packages/mwclient/client.py", line 434, in raw_api
    http_method=http_method)
  File "/home/users/federico/.local/lib/python2.7/site-packages/mwclient/client.py", line 395, in raw_call
    stream = self.connection.request(http_method, url, **args)
  File "/home/users/federico/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/users/federico/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/users/federico/.local/lib/python2.7/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='biologija.fandom.com', port=443): Read timed out. (read timeout=30)

I think it's reasonable to expect the library to try a little harder when it receives an unexpected response. (Here it's Wikia which every now and then terminates all connections for whatever reason.) Downstream code can catch such exceptions but it would be rather tedious for instance to tweak waiting times and timeouts outside of mwclient.

An approach I've used with reasponable success in dumpgenerator is to override the default HTTPAdapter as suggested in https://stackoverflow.com/a/35504626/1333493 or https://2.python-requests.org/en/master/api/#requests.adapters.HTTPAdapter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants