Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add BigQuery storage client support to DB API #36

Merged
merged 10 commits into from Feb 25, 2020

Conversation

plamut
Copy link
Contributor

@plamut plamut commented Feb 12, 2020

Closes #16.

The title says it all. Details in the ticket description.

PR checklist:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@plamut plamut added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Feb 12, 2020
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Feb 12, 2020
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General structure LGTM. Replacing self._query_data with an iterable that calls the BQ Storage API under the covers is exactly what I imagined.

Once the PR is ready, let's have @shollyman or someone he delegates to do the final review.

google/cloud/bigquery/dbapi/connection.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/connection.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
@plamut
Copy link
Contributor Author

plamut commented Feb 14, 2020

The coverage failure is due to Python 2.7 unit tests are not run on Kokoro:

Session unit-2.7 skipped: Python interpreter 2.7 not found.

The tests pass just fine locally, and the total coverage is 100%.

@plamut plamut marked this pull request as ready for review February 14, 2020 14:15
@tseaver tseaver self-requested a review February 18, 2020 18:16
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
google/cloud/bigquery/dbapi/cursor.py Outdated Show resolved Hide resolved
noxfile.py Show resolved Hide resolved
tests/unit/test_dbapi_connection.py Show resolved Hide resolved
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

1 similar comment
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Feb 19, 2020
@plamut plamut requested a review from tseaver February 19, 2020 10:50
@plamut
Copy link
Contributor Author

plamut commented Feb 20, 2020

@tseaver As a co-author of one of the changes, please just appease the CLA bot, thanks!

Copy link
Contributor

@shollyman shollyman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. We'll want to update this from v1beta1->v1 in the near future, not sure how you'd like to track that.

@plamut
Copy link
Contributor Author

plamut commented Feb 21, 2020

We'll want to update this from v1beta1->v1 in the near future, not sure how you'd like to track that.

@shollyman Track in the sense of the tasks that need to be done for the promotion to GA? A GitHub issue for it would be just fine.

@plamut
Copy link
Contributor Author

plamut commented Feb 23, 2020

@tseaver Friendly ping, the CLA bot still needs confirmation from you. Thanks!

@tseaver
Copy link
Contributor

tseaver commented Feb 25, 2020

@googlebot I signed it!

@shollyman shollyman added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Feb 25, 2020
@googlebot
Copy link

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

ℹ️ Googlers: Go here for more info.

@plamut plamut merged commit ba9b2f8 into googleapis:master Feb 25, 2020
@plamut plamut deleted the iss-16 branch February 25, 2020 18:47
@yan-hic
Copy link

yan-hic commented Jun 9, 2020

Can usage for this be updated at https://github.com/mxmzdlv/pybigquery/blob/master/README.rst#usage ?
Not sure how to enable storage API when loading in superset.

@yan-hic
Copy link

yan-hic commented Apr 7, 2021

To be closed as per googleapis/python-bigquery-sqlalchemy#61 (comment) ?

@plamut
Copy link
Contributor Author

plamut commented Apr 13, 2021

@yiga2 Sorry, I'm not following? What should be closed, this PR has been merged?

@yan-hic
Copy link

yan-hic commented Apr 13, 2021

My bad - meant to close googleapis/python-bigquery-sqlalchemy#41

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BigQuery, BigQuery Storage]: Add option to use BigQuery Storage API to download results in BigQuery DB-API
6 participants