Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consider: more extensive optional dep testing for continuous and release CI #187

Closed
shollyman opened this issue Jul 23, 2020 · 1 comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@shollyman
Copy link
Contributor

To consider:

Establish a nox session that combines optional dependencies and runs some subset of integration testing. Issue #177 revealed a case where a partially satisfied set of optional dependencies (+pyarrow, -storage) yielded issues for a consumer of the library.

Due to the combination explosion, we shouldn't run this regularly, but a periodic and release series may make more sense.

optional deps we have in setup at head at time of writing:

extras = {
    "bqstorage": [
        "google-cloud-bigquery-storage >= 1.0.0, <2.0.0dev",
        # Due to an issue in pip's dependency resolver, the `grpc` extra is not
        # installed, even though `google-cloud-bigquery-storage` specifies it
        # as `google-api-core[grpc]`. We thus need to explicitly specify it here.
        # See: https://github.com/googleapis/python-bigquery/issues/83
        "grpcio >= 1.8.2, < 2.0dev",
        "pyarrow>=0.16.0, < 2.0dev",
    ],
    "pandas": ["pandas>=0.17.1"],
    # Exclude PyArrow dependency from Windows Python 2.7.
    'pyarrow: platform_system != "Windows" or python_version >= "3.4"': [
        # Bad Linux release for 0.14.0.
        # https://issues.apache.org/jira/browse/ARROW-5868
        "pyarrow>=0.4.1, != 0.14.0"
    ],
    "tqdm": ["tqdm >= 4.0.0, <5.0.0dev"],
    "fastparquet": [
        "fastparquet",
        "python-snappy",
        # llvmlite >= 0.32.0 cannot be installed on Python 3.5 and below
        # (building the wheel fails), thus needs to be restricted.
        # See: https://github.com/googleapis/python-bigquery/issues/78
        "llvmlite <= 0.31.0",
    ],
}
@shollyman shollyman added api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Jul 23, 2020
@tswast
Copy link
Contributor

tswast commented Oct 7, 2021

#934 adds just pyarrow to one of the unit_noextras sessions. Also, since pyarrow and google-cloud-bigquery-storage are added as a required dependency in google-cloud-bigquery 3.0, hopefully this kind of problem will be less likely to happen.

@tswast tswast closed this as completed Oct 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants