samples.to_dataframe.main_test: test_query_to_dataframe failed #71

flaky-bot · 2020-09-30T09:03:24Z

This test failed!

To configure my behavior, see the Build Cop Bot documentation.

If I'm commenting on this issue too often, add the buildcop: quiet label and
I will stop commenting.

commit: 6254bf2
buildURL: Build Status, Sponge
status: failed

Test output

capsys = <_pytest.capture.CaptureFixture object at 0x7f15dace1390>
clients = (, )
def test_query_to_dataframe(capsys, clients):
    bqclient, bqstorageclient = clients

    # [START bigquerystorage_pandas_tutorial_all]
    # [START bigquerystorage_pandas_tutorial_read_query_results]
    # Download query results.
    query_string = """
    SELECT
    CONCAT(
        'https://stackoverflow.com/questions/',
        CAST(id as STRING)) as url,
    view_count
    FROM `bigquery-public-data.stackoverflow.posts_questions`
    WHERE tags like '%google-bigquery%'
    ORDER BY view_count DESC
    """

    dataframe = (
        bqclient.query(query_string)
        .result()


      .to_dataframe(bqstorage_client=bqstorageclient)


    )

main_test.py:88:

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1735: in to_dataframe

create_bqstorage_client=create_bqstorage_client,

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1549: in to_arrow

bqstorage_client=bqstorage_client

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1420: in _to_page_iterable

for item in bqstorage_download():

project_id = 'python-docs-samples-tests'

table = Table(TableReference(DatasetReference('python-docs-samples-tests', '_d5c40c0e092f2208d182028dd13910fd3f0ceb53'), 'anona8cd752bfd7a2b6c416491375c5c6d96d48ffb8b'))

bqstorage_client = <google.cloud.bigquery_storage_v1.BigQueryReadClient object at 0x7f15dacf16d8>

preserve_order = <_sre.SRE_Match object; span=(229, 237), match='ORDER BY'>

selected_fields = None

page_to_item = <function _bqstorage_page_to_arrow at 0x7f15daf65158>
def _download_table_bqstorage(
    project_id,
    table,
    bqstorage_client,
    preserve_order=False,
    selected_fields=None,
    page_to_item=None,
):
    """Use (faster, but billable) BQ Storage API to construct DataFrame."""
    if "$" in table.table_id:
        raise ValueError(
            "Reading from a specific partition is not currently supported."
        )
    if "@" in table.table_id:
        raise ValueError("Reading from a specific snapshot is not currently supported.")


  read_options = bigquery_storage_v1beta1.types.TableReadOptions()


E       AttributeError: 'NoneType' object has no attribute 'types'
.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py:620: AttributeError

The text was updated successfully, but these errors were encountered:

plamut · 2020-09-30T10:15:17Z

Will be fixed when BigQuery v2.0 is released (PR).

flaky-bot · 2020-10-17T09:09:20Z

Looks like this issue is flaky. 😟

I'm going to leave this open and stop commenting.

A human should fix and close this.

commit: a7fe762
buildURL: Build Status, Sponge
status: failed

Test output

capsys = <_pytest.capture.CaptureFixture object at 0x7fa3075a2080>
clients = (, )
def test_query_to_dataframe(capsys, clients):
    bqclient, bqstorageclient = clients

    # [START bigquerystorage_pandas_tutorial_all]
    # [START bigquerystorage_pandas_tutorial_read_query_results]
    # Download query results.
    query_string = """
    SELECT
    CONCAT(
        'https://stackoverflow.com/questions/',
        CAST(id as STRING)) as url,
    view_count
    FROM `bigquery-public-data.stackoverflow.posts_questions`
    WHERE tags like '%google-bigquery%'
    ORDER BY view_count DESC
    """

    dataframe = (
        bqclient.query(query_string)
        .result()


      .to_dataframe(bqstorage_client=bqstorageclient)


    )

main_test.py:88:

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1735: in to_dataframe

create_bqstorage_client=create_bqstorage_client,

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1549: in to_arrow

bqstorage_client=bqstorage_client

.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/table.py:1420: in _to_page_iterable

for item in bqstorage_download():

project_id = 'python-docs-samples-tests'

table = Table(TableReference(DatasetReference('python-docs-samples-tests', '_d5c40c0e092f2208d182028dd13910fd3f0ceb53'), 'anon6c6d4955a20346676de474ebd41d97588b6b20e1'))

bqstorage_client = <google.cloud.bigquery_storage_v1.BigQueryReadClient object at 0x7fa307588550>

preserve_order = <_sre.SRE_Match object; span=(229, 237), match='ORDER BY'>

selected_fields = None

page_to_item = <function _bqstorage_page_to_arrow at 0x7fa307bdbd08>
def _download_table_bqstorage(
    project_id,
    table,
    bqstorage_client,
    preserve_order=False,
    selected_fields=None,
    page_to_item=None,
):
    """Use (faster, but billable) BQ Storage API to construct DataFrame."""
    if "$" in table.table_id:
        raise ValueError(
            "Reading from a specific partition is not currently supported."
        )
    if "@" in table.table_id:
        raise ValueError("Reading from a specific snapshot is not currently supported.")


  read_options = bigquery_storage_v1beta1.types.TableReadOptions()


E       AttributeError: 'NoneType' object has no attribute 'types'
.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/_pandas_helpers.py:620: AttributeError

tswast · 2020-10-20T21:16:12Z

I tested locally and it passes.

$ nox
No user noxfile_config found: detail: No module named 'noxfile_config'
nox > Running session lint
nox > Creating virtual environment (virtualenv) using python in .nox/lint
nox > pip install flake8 flake8-import-order
nox > flake8 --show-source --builtin=gettext --max-complexity=20 --import-order-style=google --exclude=.nox,.cache,env,lib,generated_pb2,*_pb2.py,*_pb2_grpc.py --ignore=E121,E123,E126,E203,E226,E24,E266,E501,E704,W503,W504,I202 --max-line-length=88 --application-import-names jupyter_test,__init__,.nox,main_test,noxfile .
nox > Session lint was successful.
nox > Running session py-2.7
nox > Creating virtual environment (virtualenv) using python2.7 in .nox/py-2-7
nox > Session py-2.7 skipped: SKIPPED: 2.7 tests are disabled for this sample..
nox > Running session py-3.6
nox > Creating virtual environment (virtualenv) using python3.6 in .nox/py-3-6
nox > pip install -r requirements.txt
nox > pip install -r requirements-test.txt
nox > pytest --junitxml=sponge_log.xml
==================================================== test session starts ====================================================
platform darwin -- Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/swast/src/python-bigquery-storage
collected 5 items                                                                                                           

jupyter_test.py ..                                                                                                    [ 40%]
main_test.py ...                                                                                                      [100%]

===================================================== warnings summary ======================================================
.nox/py-3-6/lib/python3.6/site-packages/_pytest/junitxml.py:446
  /Users/swast/src/python-bigquery-storage/samples/to_dataframe/.nox/py-3-6/lib/python3.6/site-packages/_pytest/junitxml.py:446: PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0. See:
    https://docs.pytest.org/en/stable/deprecations.html#junit-family-default-value-change-to-xunit2
  for more information.
    _issue_warning_captured(deprecated.JUNIT_XML_DEFAULT_FAMILY, config.hook, 2)

samples/to_dataframe/jupyter_test.py::test_jupyter_small_query
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
    return f(*args, **kwds)

samples/to_dataframe/jupyter_test.py::test_jupyter_small_query
samples/to_dataframe/jupyter_test.py::test_jupyter_tutorial
  /Users/swast/src/python-bigquery-storage/samples/to_dataframe/.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/magics/magics.py:497: DeprecationWarning: Deprecated option --use_bqstorage_api, the BigQuery Storage API is already used by default.
    category=DeprecationWarning,

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------- generated xml file: /Users/swast/src/python-bigquery-storage/samples/to_dataframe/sponge_log.xml --------------
========================================= 5 passed, 4 warnings in 63.39s (0:01:03) ==========================================

No idea why the build is still failing.

tswast · 2020-10-20T21:19:46Z

Also passes with INSTALL_LIBRARY_FROM_SOURCE.

$ export INSTALL_LIBRARY_FROM_SOURCE=True

$ nox -s py-3.6
No user noxfile_config found: detail: No module named 'noxfile_config'
nox > Running session py-3.6
nox > Creating virtual environment (virtualenv) using python3.6 in .nox/py-3-6
nox > pip install -r requirements.txt
nox > pip install -r requirements-test.txt
nox > pip install -e /Users/swast/src/python-bigquery-storage
nox > pytest --junitxml=sponge_log.xml
==================================================== test session starts ====================================================
platform darwin -- Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/swast/src/python-bigquery-storage
collected 5 items                                                                                                           

jupyter_test.py ..                                                                                                    [ 40%]
main_test.py ...                                                                                                      [100%]

===================================================== warnings summary ======================================================
.nox/py-3-6/lib/python3.6/site-packages/_pytest/junitxml.py:446
  /Users/swast/src/python-bigquery-storage/samples/to_dataframe/.nox/py-3-6/lib/python3.6/site-packages/_pytest/junitxml.py:446: PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0. See:
    https://docs.pytest.org/en/stable/deprecations.html#junit-family-default-value-change-to-xunit2
  for more information.
    _issue_warning_captured(deprecated.JUNIT_XML_DEFAULT_FAMILY, config.hook, 2)

samples/to_dataframe/jupyter_test.py::test_jupyter_small_query
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
    return f(*args, **kwds)

samples/to_dataframe/jupyter_test.py::test_jupyter_small_query
samples/to_dataframe/jupyter_test.py::test_jupyter_tutorial
  /Users/swast/src/python-bigquery-storage/samples/to_dataframe/.nox/py-3-6/lib/python3.6/site-packages/google/cloud/bigquery/magics/magics.py:497: DeprecationWarning: Deprecated option --use_bqstorage_api, the BigQuery Storage API is already used by default.
    category=DeprecationWarning,

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------- generated xml file: /Users/swast/src/python-bigquery-storage/samples/to_dataframe/sponge_log.xml --------------
============================================== 5 passed, 4 warnings in 49.70s ===============================================
nox > Session py-3.6 was successful.
(googleapis)

flaky-bot bot added buildcop: issue priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Sep 30, 2020

product-auto-label bot added api: bigquerystorage Issues related to the googleapis/python-bigquery-storage API. samples Issues that are directly related to samples. labels Sep 30, 2020

yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels Oct 5, 2020

meredithslota added priority: p2 Moderately-important priority. Fix may not be included in next release. and removed priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Oct 5, 2020

tswast mentioned this issue Oct 15, 2020

docs: update to_dataframe sample to latest dependencies #72

Merged

4 tasks

tswast closed this as completed in #72 Oct 16, 2020

flaky-bot bot reopened this Oct 17, 2020

flaky-bot bot added the buildcop: flaky label Oct 17, 2020

tswast closed this as completed Oct 21, 2020

JustinBeckwith assigned tswast Feb 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

samples.to_dataframe.main_test: test_query_to_dataframe failed #71

samples.to_dataframe.main_test: test_query_to_dataframe failed #71

flaky-bot bot commented Sep 30, 2020

plamut commented Sep 30, 2020

flaky-bot bot commented Oct 17, 2020

tswast commented Oct 20, 2020

tswast commented Oct 20, 2020

samples.to_dataframe.main_test: test_query_to_dataframe failed #71

samples.to_dataframe.main_test: test_query_to_dataframe failed #71

Comments

flaky-bot bot commented Sep 30, 2020

plamut commented Sep 30, 2020

flaky-bot bot commented Oct 17, 2020

tswast commented Oct 20, 2020

tswast commented Oct 20, 2020