Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: unnest failed in some cases (with table references failed when there were no other references to refrenced tables in a query) #290

Merged
merged 11 commits into from Aug 25, 2021

Conversation

jimfulton
Copy link
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #283 🦕

@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Aug 20, 2021
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label Aug 20, 2021
@jimfulton jimfulton marked this pull request as ready for review August 20, 2021 20:44
@jimfulton jimfulton requested a review from a team as a code owner August 20, 2021 20:44
@jimfulton
Copy link
Contributor Author

Reviewer:

This PR is explained pretty well, I think, by the comment here:
https://github.com/googleapis/python-bigquery-sqlalchemy/pull/290/files#diff-efe1987a73ae5b54aa817dc8c70541025df89d9d08601c8ab33fa84ad8c09de7R254-R272

Mostly unrelated, I added the missing ARRAY type to the list of types, to facilitate a test. I took the opportunity to sort the types, which contributes a quite a few changed lines, but the change is trivial.

@jimfulton jimfulton changed the title fix: unnest with table references failed when there were no other references to refrenced tables fix: unnest failed in some cases (with table references failed when there were no other references to refrenced tables in a query) Aug 23, 2021
}

# By convention, dialect-provided types are spelled with all upper case.
STRING = _type_map["STRING"]
BOOL = _type_map["BOOL"]
ARRAY = _type_map["ARRAY"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the new array type count as a feature? Perhaps it'd look better in the changelog as a separate PR?

If it is a feature and it doesn't make sense to split out, we could also try the multi-change commit message feature when we squash and merge this: https://github.com/googleapis/release-please#what-if-my-pr-contains-multiple-fixes-or-features

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a new feature. It was just missing from the types list.

SQLAlchemy has basic array support that works with BigQuery (thanks to some previous effort on our part).

sqlalchemy_bigquery/base.py Outdated Show resolved Hide resolved
sqlalchemy_bigquery/base.py Show resolved Hide resolved
@@ -34,6 +34,7 @@
import datetime
import decimal

sqlalchemy_version_info = tuple(map(int, sqlalchemy.__version__.split(".")))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works, though I've been trying to use https://packaging.pypa.io/en/latest/version.html which is the canonical version parser. We already have it pulled in via a transitive dependency through setuptools, I believe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I'll switch to that.

WRT how we get it, IMO, anything we import should be a direct dependency.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I'm fine explicitly including it in our dependencies / test dependencies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jimfulton
Copy link
Contributor Author

@tswast are you cool with this? :)

@jimfulton
Copy link
Contributor Author

I have to more small PRs coming that will build on this.

@jimfulton jimfulton merged commit 9b5b002 into googleapis:master Aug 25, 2021
@jimfulton jimfulton deleted the fix-unnest-283 branch August 25, 2021 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

unnest isn't handled correctly
2 participants