Storage: Dependency check on 'requests' raises dependency inconsistencies

[EpicWink]

When importing google.cloud.storage, a pkg_resources.ContextualVersionConflict is raised (causing an ImportError to be raised by google-resumable-media-python), as requests requires urllib3<1.25 but urllib3 v1.25 is installed. This is due to a pkg_resources.require('requests >= 2.18.0') run-time check.

The best solution would be to remove that check, and instead check for the required features of requests at import time.

• Issue in requests: https://github.com/kennethreitz/requests/issues/5067
• Issue in google-resumable-media-python: Dependency check on 'requests' raises dependency inconsistencies google-resumable-media-python#64

Also affects google-cloud-biquery probably.

Environment details

OS: Ubuntu 18.10.1 (irrelevant)
Python: 3.6.8 (irrelevant)
google-cloud-storage: 1.15.0
requests: 2.21.0
urllib3: 1.25

Steps to reproduce

pip3 install sentry-sdk google-cloud-storage

>>> from google.cloud import storage

sentry-sdk is installed as it pulls the latest version of urllib3 before the dependencies of google-cloud-storage are analysed.

Stack trace

env/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
Traceback (most recent call last):
  File "env/lib/python3.6/site-packages/google/resumable_media/requests/__init__.py", line 665, in <module>
    pkg_resources.require('requests >= 2.18.0')
  File "env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 892, in require
    needed = self.resolve(parse_requirements(requirements))
  File "env/lib/python3.6/site-packages/pkg_resources/__init__.py", line 783, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (urllib3 1.25 (env/lib/python3.6/site-packages), Requirement.parse('urllib3<1.25,>=1.21.1'), {'requests'})

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "env/lib/python3.6/site-packages/google/cloud/storage/__init__.py", line 39, in <module>
    from google.cloud.storage.blob import Blob
  File "env/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 44, in <module>
    from google.resumable_media.requests import ChunkedDownload
  File "env/lib/python3.6/site-packages/google/resumable_media/requests/__init__.py", line 673, in <module>
    six.raise_from(new_exc, caught_exc)
  File "<string>", line 3, in raise_from
ImportError: ``requests >= 2.18.0`` is required by the ``google.resumable_media.requests`` subpackage.
It can be installed via
    pip install google-resumable-media[requests].

[tseaver]

I cannot reproduce this in a clean virtual environment:

$ python3.6 -m venv /tmp/gcp/7855
$ /tmp/gcp/7855/bin/pip install --upgrade setuptools pip wheel
...
Successfully installed pip-19.1.1 setuptools-41.0.1 wheel-0.33.4
$ /tmp/gcp/7855/bin/pip install sentry-sdk google-cloud-storage
Successfully installed cachetools-3.1.0 certifi-2019.3.9 chardet-3.0.4 google-api-core-1.11.0 google-auth-1.6.3 google-cloud-core-1.0.0 google-cloud-storage-1.16.0 google-resumable-media-0.3.2 googleapis-common-protos-1.6.0 idna-2.8 protobuf-3.7.1 pyasn1-0.4.5 pyasn1-modules-0.2.5 pytz-2019.1 requests-2.22.0 rsa-4.0 sentry-sdk-0.8.0 six-1.12.0 urllib3-1.25.3
$ /tmp/gcp/7855/bin/python -c "from google.cloud import storage; print(storage.__version__)"
1.16.0

Output from pip freeze in that environment:

cachetools==3.1.0
certifi==2019.3.9
chardet==3.0.4
google-api-core==1.11.0
google-auth==1.6.3
google-cloud-core==1.0.0
google-cloud-storage==1.16.0
google-resumable-media==0.3.2
googleapis-common-protos==1.6.0
idna==2.8
protobuf==3.7.1
pyasn1==0.4.5
pyasn1-modules==0.2.5
pytz==2019.1
requests==2.22.0
rsa==4.0
sentry-sdk==0.8.0
six==1.12.0
urllib3==1.25.3

[tseaver]

I'm presuming that the release of requests 2.22.0 fixed this issue.

[EpicWink]

You're right, requests v2.22.0 did fix the issue, however the issue will arise again when urllib3 v1.26 is released. My request for comment is on the existence of the runtime version check on requests