Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve InvalidSchema exception by adding specific fields that do not match #349

Open
artemrys opened this issue Dec 6, 2020 · 4 comments
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@artemrys
Copy link

artemrys commented Dec 6, 2020

I am talking about this InvalidSchema exception.

It may be useful to show at most 3 (for example) fields that have different types. It should help to debug a type mismatch faster.

@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Dec 17, 2020
@tswast
Copy link
Collaborator

tswast commented Dec 17, 2020

Great suggestion. I think we do a "set difference" in order to discover schema mismatch, so this should be possible to figure out and include in the exception.

@artemrys
Copy link
Author

Great, I will start working on it then.

@tswast
Copy link
Collaborator

tswast commented Dec 23, 2021

I'm a bit wary of including this information in the actual exception message due to potential privacy issues around logging. Instead, I'm adding local_schema and remote_schema properties to InvalidSchema in #449

@tswast
Copy link
Collaborator

tswast commented Jul 26, 2022

So long as we leave out description (and other column metadata like IAM roles), which can contain arbitrary text (possibly, though unlikely to be PII) I do think it would be an improved experience if the string representation of InvalidSchema showed the diff of field names and the types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants