Bad error handling in requests in python 3 [During handling of the above exception, another exception occurred]

[Gonzalliz]

request library with Python3 handles exceptions in a pretty bad way. In python2 this works fine. I tried to request GitHub using requests.get('https://api.github.com') using both python2 installed requests library and python3 installed requests library. They perform differently when the requests are made during network is down. (any exception is handled differently). Try installing requests in both python2 and python3 and you can recreate the issue by removing internet connection when making the requests. Code I used
import requests
requests.get('https://api.github.com')

I have installed requests==2.5.3 in my system and I tried upgrading the version to requests==2.22.0 but still problem persists.

Expected Result

In python2 it gives a cleaner exception like the following
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 65, in get return request('get', url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 49, in request response = session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 461, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 415, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', gaierror(-2, 'Name or service not known'))

Actual Result

`
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 334, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f2ab2562d68>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f2ab2562d68>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f2ab2562d68>: Failed to establish a new connection: [Errno -2] Name or service not known',))

`

Reproduction Steps

import requests

System Information

$ python -m requests.help

Python 2.7.15+ (Python2 version)
requests==2.5.3 (Requests version in python2)
=============================
Python 3.6.9 (Python3 version)
requests==2.5.3(Requests version in python3)
=============================
OS: Ubuntu 18.04.3 LTS

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

[mbanders]

I ran into this problem too, but I found a work-around for prettier output. Notice the "from None" in the script here:

import requests
url = 'http://10.0.0.12/example/'
try:
    requests.get(url, timeout=2)
except requests.exceptions.RequestException:
    raise Exception('Failed to connect to %s' % url) from None

If you run the above script the output is much cleaner:

$ python example.py
Traceback (most recent call last):
  File "example.py", line 6, in <module>
    raise Exception('Failed to connect to %s' % url) from None
Exception: Failed to connect to http://10.0.0.12/example/

You can only do the from None in Python3, and I think it comes from this PEP: https://www.python.org/dev/peps/pep-0409/

[Gonzalliz]

I ran into this problem too, but I found a work-around for prettier output. Notice the "from None" in the script here:

import requests
url = 'http://10.0.0.12/example/'
try:
    requests.get(url, timeout=2)
except requests.exceptions.RequestException:
    raise Exception('Failed to connect to %s' % url) from None

If you run the above script the output is much cleaner:

$ python example.py
Traceback (most recent call last):
  File "example.py", line 6, in <module>
    raise Exception('Failed to connect to %s' % url) from None
Exception: Failed to connect to http://10.0.0.12/example/

You can only do the from None in Python3, and I think it comes from this PEP: https://www.python.org/dev/peps/pep-0409/

[shaderecker]

I just stumbled upon the same issue, is there any way with this workaround to continue after the error?
My program should not exit after the raised exception but I didn't find any way to do that.

[PJJUNE2]

For anyone who has this issue please check your VPN!!!
I have same problems with request.get in every website utill I trun off my VPN

[jakermx]

While VPN handling is a OS dependant respoinsability, I would better suggest the use of local proxy triggered by the IPSEC P2 connected status....so that way, your code ensures the correct L3 routing tables.

New Linux kernel, has a new behaivor that drops from memory the routing table when a new route is added, making the last well know working stack unusable...so is you thread your proxy based on your OS routing table status....it will work crystal

[sethmlarson]

I believe this is a side-effect of retries in urllib3, hence Requests can't do anything about this. Closing this for now.