Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New 'page_size' argument breaks unit tests under Python 3.6 #529

Closed
tseaver opened this issue Aug 2, 2021 · 0 comments · Fixed by #530
Closed

New 'page_size' argument breaks unit tests under Python 3.6 #529

tseaver opened this issue Aug 2, 2021 · 0 comments · Fixed by #530
Assignees
Labels
api: storage Issues related to the googleapis/python-storage API. type: process A process-related concern. May include testing, release, or the like.

Comments

@tseaver
Copy link
Contributor

tseaver commented Aug 2, 2021

PR #520 added passing page_size to the google.api_core.page_iterator.HTTPIterator constructor, which is a feature added only in google-api-core 1.29.0. testing/constraints-3.6.txt pins google-cloud-core==1.6.0, the minimum version specified in setup.py, but does not pin google-api-core, which results in:

$ git remote -v
origin	git@github.com:googleapis/python-storage (fetch)
origin	git@github.com:googleapis/python-storage (push)

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

$ git log -1
commit 4abb40310eca7ec45afc4bc5e4dfafbe083e74d2 (HEAD -> master, origin/master, origin/HEAD)
Author: Tres Seaver <tseaver@palladion.com>
Date:   Tue Jul 27 13:43:42 2021 -0400

    fix: make 'requests.exceptions.ChunkedEncodingError retryable by default (#526)
    
    Closes #525.

$ .nox/unit-3-6/bin/pip list | grep google
google-api-core          1.27.0
google-auth              1.24.0
google-cloud-core        1.6.0
google-cloud-storage     1.41.1   /path/to/python-storage
google-crc32c            1.1.2
google-resumable-media   1.3.0
googleapis-common-protos 1.53.0

$ nox -re unit-3.6
nox > Running session unit-3.6
nox > Creating virtual environment (virtualenv) using python3.6 in .nox/unit-3-6
nox > python -m pip install mock pytest pytest-cov -c /home/tseaver/projects/agendaless/Google/src/python-storage/testing/constraints-3.6.txt
nox > python -m pip install -e . -c /home/tseaver/projects/agendaless/Google/src/python-storage/testing/constraints-3.6.txt
nox > py.test --quiet --cov=google.cloud.storage --cov=google.cloud --cov=tests.unit --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=0 tests/unit
..............................................s......................... [  6%]
........................................................................ [ 13%]
........................................................................ [ 20%]
........................................................................ [ 27%]
........................................................................ [ 34%]
........................................................................ [ 41%]
........................................................................ [ 48%]
........................................................................ [ 55%]
........................................................................ [ 62%]
........................................................................ [ 69%]
........................................................................ [ 76%]
.........................FF............................................. [ 83%]
........................................................................ [ 90%]
........................................................................ [ 97%]
.....................                                                    [100%]Coverage.py warning: Module google.cloud was previously imported, but not measured (module-not-measured)

=================================== FAILURES ===================================
__________________ TestClient.test__list_resource_w_defaults ___________________

self = <tests.unit.test_client.TestClient testMethod=test__list_resource_w_defaults>

    def test__list_resource_w_defaults(self):
        import functools
        from google.api_core.page_iterator import HTTPIterator
        from google.api_core.page_iterator import _do_nothing_page_start
    
        project = "PROJECT"
        path = "/path/to/list/resource"
        item_to_value = mock.Mock(spec=[])
        credentials = _make_credentials()
        client = self._make_one(project=project, credentials=credentials)
        connection = client._base_connection = _make_connection()
    
>       iterator = client._list_resource(path=path, item_to_value=item_to_value,)

tests/unit/test_client.py:485: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <google.cloud.storage.client.Client object at 0x7efcfb9f3b38>
path = '/path/to/list/resource', item_to_value = <Mock id='139625018221344'>
page_token = None, max_results = None, extra_params = None
page_start = <function _do_nothing_page_start at 0x7efcfc69fae8>
page_size = None, timeout = 60
retry = <google.api_core.retry.Retry object at 0x7efcfc756c50>

    def _list_resource(
        self,
        path,
        item_to_value,
        page_token=None,
        max_results=None,
        extra_params=None,
        page_start=page_iterator._do_nothing_page_start,
        page_size=None,
        timeout=_DEFAULT_TIMEOUT,
        retry=DEFAULT_RETRY,
    ):
        api_request = functools.partial(
            self._connection.api_request, timeout=timeout, retry=retry
        )
        return page_iterator.HTTPIterator(
            client=self,
            api_request=api_request,
            path=path,
            item_to_value=item_to_value,
            page_token=page_token,
            max_results=max_results,
            extra_params=extra_params,
            page_start=page_start,
>           page_size=page_size,
        )
E       TypeError: __init__() got an unexpected keyword argument 'page_size'

google/cloud/storage/client.py:403: TypeError
__________________ TestClient.test__list_resource_w_explicit ___________________

self = <tests.unit.test_client.TestClient testMethod=test__list_resource_w_explicit>

    def test__list_resource_w_explicit(self):
        import functools
        from google.api_core.page_iterator import HTTPIterator
    
        project = "PROJECT"
        path = "/path/to/list/resource"
        item_to_value = mock.Mock(spec=[])
        page_token = "PAGE-TOKEN"
        max_results = 47
        extra_params = {"foo": "Foo"}
        page_start = mock.Mock(spec=[])
        credentials = _make_credentials()
        client = self._make_one(project=project, credentials=credentials)
        connection = client._base_connection = _make_connection()
    
        iterator = client._list_resource(
            path=path,
            item_to_value=item_to_value,
            page_token=page_token,
            max_results=max_results,
            extra_params=extra_params,
>           page_start=page_start,
        )

tests/unit/test_client.py:523: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <google.cloud.storage.client.Client object at 0x7efcfbdfe470>
path = '/path/to/list/resource', item_to_value = <Mock id='139625022614160'>
page_token = 'PAGE-TOKEN', max_results = 47, extra_params = {'foo': 'Foo'}
page_start = <Mock id='139625022612368'>, page_size = None, timeout = 60
retry = <google.api_core.retry.Retry object at 0x7efcfc756c50>

    def _list_resource(
        self,
        path,
        item_to_value,
        page_token=None,
        max_results=None,
        extra_params=None,
        page_start=page_iterator._do_nothing_page_start,
        page_size=None,
        timeout=_DEFAULT_TIMEOUT,
        retry=DEFAULT_RETRY,
    ):
        api_request = functools.partial(
            self._connection.api_request, timeout=timeout, retry=retry
        )
        return page_iterator.HTTPIterator(
            client=self,
            api_request=api_request,
            path=path,
            item_to_value=item_to_value,
            page_token=page_token,
            max_results=max_results,
            extra_params=extra_params,
            page_start=page_start,
>           page_size=page_size,
        )
E       TypeError: __init__() got an unexpected keyword argument 'page_size'

google/cloud/storage/client.py:403: TypeError


=========================== short test summary info ============================
FAILED tests/unit/test_client.py::TestClient::test__list_resource_w_defaults
FAILED tests/unit/test_client.py::TestClient::test__list_resource_w_explicit
2 failed, 1026 passed, 1 skipped in 19.16s
nox > Command py.test --quiet --cov=google.cloud.storage --cov=google.cloud --cov=tests.unit --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=0 tests/unit failed with exit code 1
nox > Session unit-3.6 failed.

The fix would be to add ranges for google-api-core and google-auth (another transitive dependency) to setup.py, and then pin the minimum versions for those ranges in testing/constraints-3.6.txt.

@tseaver tseaver added the type: process A process-related concern. May include testing, release, or the like. label Aug 2, 2021
@tseaver tseaver self-assigned this Aug 2, 2021
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Aug 2, 2021
tseaver added a commit that referenced this issue Aug 2, 2021
Transitive dependency resolution breaks unit tests under Python 3.6
following PR #520.

Pin minima for those ranges explicitly when testing under Python 3.6.

Closes #529.
gcf-merge-on-green bot pushed a commit that referenced this issue Aug 5, 2021
#530)

Transitive dependency resolution breaks unit tests under Python 3.6
following PR #520.

Pin minima for those ranges explicitly when testing under Python 3.6.

Closes #529.
cojenco pushed a commit to cojenco/python-storage that referenced this issue Oct 13, 2021
googleapis#530)

Transitive dependency resolution breaks unit tests under Python 3.6
following PR googleapis#520.

Pin minima for those ranges explicitly when testing under Python 3.6.

Closes googleapis#529.
cojenco pushed a commit to cojenco/python-storage that referenced this issue Oct 13, 2021
googleapis#530)

Transitive dependency resolution breaks unit tests under Python 3.6
following PR googleapis#520.

Pin minima for those ranges explicitly when testing under Python 3.6.

Closes googleapis#529.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/python-storage API. type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant