Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bigquery): allow construction of jobs from other projects #5048

Merged
merged 5 commits into from Oct 31, 2021

Conversation

shollyman
Copy link
Contributor

Adds a JobfromProject() method to allow users to get metadata from a
job that was created within another project.

Fixes: #4228

Adds a `JobfromProject()` method to allow users to get metadata from a
job that was created within another project.

Fixes: googleapis#4228
@shollyman shollyman requested a review from a team October 29, 2021 18:14
@shollyman shollyman requested a review from a team as a code owner October 29, 2021 18:14
@shollyman shollyman requested a review from tswast October 29, 2021 18:14
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Oct 29, 2021
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the BigQuery API. label Oct 29, 2021
// JobFromProject creates a Job which refers to an existing BigQuery job. The job
// need not have been created by this package, nor does it need to reside within the same
// project or location as the instantiated client.
func (c *Client) JobFromProject(ctx context.Context, projectID, jobID, location string) (j *Job, err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine to me. Another option would be to have a config struct here and let users selectively override particular parameters than what they have set with their client.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern is used mostly because its in line with how we expose similar out-of-project resources like datasets. There's a minor preposition inconsistency though, in jobs we do JobFrom, but datasets are DatasetInProject. Seemed better to be consistent with the other job constructor names.

Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design LGTM. Are there other places that use c.projectID that need to be updated? Might be good to see a unit test or two that verifies when the job project != client project sets the expect HTTP path.

@shollyman
Copy link
Contributor Author

Design LGTM. Are there other places that use c.projectID that need to be updated? Might be good to see a unit test or two that verifies when the job project != client project sets the expect HTTP path.

Other callsites that reference the client's projectID directly is job insertion, either the standard jobs.insert or the optimized jobs.query path. There's other things like dataset iterator Datasets(), but under the hood it calls DatasetsInProject() which allows remote enumeration.

@shollyman
Copy link
Contributor Author

Getting this to a unit test is surprisingly annoying given the lack of intermediate interfaces, so I wrote a small integration test for now.

@shollyman shollyman added the automerge Merge the pull request once unit tests and other checks pass. label Oct 31, 2021
@gcf-merge-on-green gcf-merge-on-green bot merged commit 6d07eca into googleapis:master Oct 31, 2021
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Oct 31, 2021
shollyman added a commit to shollyman/google-cloud-go that referenced this pull request Nov 1, 2021
…ature

As part of the changes for
googleapis#5048 one callsite of
getJobInternal was missed.  Normally this would easily get identified
due to the change in signature, but getJobInternal has a set of expected
string arguments, followed by variadic string args.  This got picked up
by integration testing, but I failed to recall that presubmit doesn't run
integration tests so it was caught after submit.

Mostly this one's a cautionary tale for having a mix of mandatory and
variadic functions that share the same type.

Fixes: googleapis#5058
shollyman added a commit that referenced this pull request Nov 1, 2021
…ature (#5059)

As part of the changes for
#5048 one callsite of
getJobInternal was missed.  Normally this would easily get identified
due to the change in signature, but getJobInternal has a set of expected
string arguments, followed by variadic string args.  This got picked up
by integration testing, but I failed to recall that presubmit doesn't run
integration tests so it was caught after submit.

Mostly this one's a cautionary tale for having a mix of mandatory and
variadic functions that share the same type.

Fixes: #5058
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bigquery: allow construction of arbitrary job references
3 participants