Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SchemaField cannot initialize from __repr__ #1041

Closed
WesRoach opened this issue Nov 1, 2021 · 4 comments · Fixed by #1046
Closed

SchemaField cannot initialize from __repr__ #1041

WesRoach opened this issue Nov 1, 2021 · 4 comments · Fixed by #1046
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@WesRoach
Copy link

WesRoach commented Nov 1, 2021

Environment details

  • OS type and version: macOS Big Sur 11.6.1
  • Python version: 3.9.7
  • pip version: 21.3.1
  • google-cloud-bigquery version: 2.29.0

Description

Prior versions of google-cloud-bigquery (2.3.1) allowed SchemaField instantiation based on an existing SchemaField __repr__.

Use Case

  1. System generates list of SchemaField and passes it through a Jinja template as part of building hundreds of Airflow DAGs.
  2. Jinja prints __repr__ from the SchemaField into the templated .py.
  3. Airflow picks up the .py files and executes them.
  4. Execution results in the below error.

Code example

In version 2.3.1

from google.cloud.bigquery import SchemaField
from google.cloud import bigquery
bigquery.__version__
'2.3.1'

# This works
field = SchemaField('name', 'STRING', 'REQUIRED', 'Name') 
field.__repr__()
"SchemaField('name', 'STRING', 'REQUIRED', 'Name', (), ())"
# Jinja prints ^^ to a .py file
# When the file is executed the SchemaField is created without issue

Version 2.29.0:

from google.cloud.bigquery import SchemaField
from google.cloud import bigquery
bigquery.__version__
'2.29.0'

# This raises an error for the tuple in the last field
field = SchemaField('name', 'STRING', 'REQUIRED', 'Name') 
field.__repr__()
"SchemaField('name', 'STRING', 'REQUIRED', 'Name', (), ())"
# Jinja prints ^^ to a .py file
# When the file is executed, an error is raised:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wes.roach/.pyenv/versions/upgr-dbt/lib/python3.9/site-packages/google/cloud/bigquery/schema.py", line 125, in __init__
    policy_tags.to_api_repr() if policy_tags is not None else None
AttributeError: 'tuple' object has no attribute 'to_api_repr'

Setting the policy_tags to None still results in ().

field = SchemaField('name', 'STRING', 'REQUIRED', 'Name', (), None)
field.__repr__()
"SchemaField('name', 'STRING', 'REQUIRED', 'Name', (), ())"
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Nov 1, 2021
@tswast tswast added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Nov 2, 2021
@tswast
Copy link
Contributor

tswast commented Nov 2, 2021

@WesRoach Thanks for the report! This does appear to be a bug introduced when policy tags were added.

@tswast tswast added the priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. label Nov 2, 2021
@tswast
Copy link
Contributor

tswast commented Nov 2, 2021

@plamut Potential project for you if you have cycles? It'd be good to get this in v2.

@plamut
Copy link
Contributor

plamut commented Nov 3, 2021

@tswast I'll take a look, yes.

@plamut
Copy link
Contributor

plamut commented Nov 4, 2021

This does appear to be a bug introduced when policy tags were added.

Confirmed, the regression was caused by #983.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants