Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigquery): export HivePartitioningOptions in load job configurations #3877

Merged
merged 6 commits into from Apr 2, 2021

Conversation

shollyman
Copy link
Contributor

Fixes: #3876

@shollyman shollyman requested a review from a team as a code owner March 31, 2021 19:22
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the BigQuery API. label Mar 31, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Mar 31, 2021
@shollyman shollyman requested a review from hongalex March 31, 2021 19:36
Copy link
Member

@hongalex hongalex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: conventional commits for commit message

feat(bigquery): export ....

@shollyman shollyman changed the title bigquery: export HiveParititioningOptions in load job configurations feat(bigquery): export HiveParititioningOptions in load job configurations Apr 1, 2021
@shollyman
Copy link
Contributor Author

nit: conventional commits for commit message

Weird that the conventional commit bot didn't seem to have issue with it. Updated.

@hongalex
Copy link
Member

hongalex commented Apr 1, 2021

nit: conventional commits for commit message

Weird that the conventional commit bot didn't seem to have issue with it. Updated.

I think it just checks for the presence of any word before the colon, which the original go commit format satisfies

@shollyman shollyman changed the title feat(bigquery): export HiveParititioningOptions in load job configurations feat(bigquery): export HivePartitioningOptions in load job configurations Apr 1, 2021
bigquery/load.go Outdated Show resolved Hide resolved
bigquery/load.go Outdated Show resolved Hide resolved
bigquery/load.go Outdated
@@ -111,6 +118,9 @@ func bqToLoadConfig(q *bq.JobConfiguration, c *Client) *LoadConfig {
fc = &s.FileConfig
lc.Src = s
}
if q.Load.HivePartitioningOptions != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this still needs an update?

RequirePartitionFilter: true,
},
},
want: func() *bq.Job {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯

This took me a second.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have no idea how much I wish we weren't in the business of mapping back and forth to discovery representations for complex resources like job configs.

shollyman and others added 3 commits April 2, 2021 09:12
Co-authored-by: Tyler Bui-Palsulich <26876514+tbpg@users.noreply.github.com>
bigquery/load.go Outdated
@@ -111,6 +118,9 @@ func bqToLoadConfig(q *bq.JobConfiguration, c *Client) *LoadConfig {
fc = &s.FileConfig
lc.Src = s
}
if q.Load.HivePartitioningOptions != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this still needs an update?

@shollyman shollyman merged commit 7c759be into googleapis:master Apr 2, 2021
@shollyman shollyman deleted the fr-hiveloads branch April 2, 2021 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bigquery: expose hive configuration options in LoadConfig
3 participants