Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BigQuery.listJobs is throwing NPE #186

Closed
shevek opened this issue Feb 23, 2020 · 6 comments · May be fixed by #2241
Closed

BigQuery.listJobs is throwing NPE #186

shevek opened this issue Feb 23, 2020 · 6 comments · May be fixed by #2241
Assignees
@pmakani
Labels
api: bigquery Issues related to the googleapis/java-bigquery API. needs more info This issue needs more information from the customer to proceed. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@shevek
Copy link

shevek commented Feb 23, 2020

Follow on from #17 (which still exists): listJobs is ALSO throwing NPE:

java.lang.NullPointerException: null
        at com.google.cloud.bigquery.JobStatistics.fromPb(JobStatistics.java:1183)
        at com.google.cloud.bigquery.JobInfo$BuilderImpl.<init>(JobInfo.java:154)
        at com.google.cloud.bigquery.Job.fromPb(Job.java:485)
        at com.google.cloud.bigquery.BigQueryImpl$32.apply(BigQueryImpl.java:1127)
        at com.google.cloud.bigquery.BigQueryImpl$32.apply(BigQueryImpl.java:1124)
        at com.google.common.collect.Iterators$6.transform(Iterators.java:783)
        at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)

This is clearly dependent on the data the server is sending; in your tests you are not reproducing the server-side situation that we have; I don't know what that server side situation is, but I know it exists. In both this case and case #17, the client needs to be much more defensively programmed.

    JobConfiguration jobConfigPb = jobPb.getConfiguration(); // RETURNS NULL
    com.google.api.services.bigquery.model.JobStatistics statisticPb = jobPb.getStatistics();
    if (jobConfigPb.getLoad() != null) { // NPE
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/java-bigquery API. label Feb 23, 2020
@shevek
Copy link
Author

shevek commented Feb 23, 2020

I'm going to take a wild guess here and say that listJobs can't handle jobs which were (a) dry-run, and (b) returned an error. Not totally sure. In any case, both this issue and #17 need some good client side defensive programming, because the situations ARE arising, and they are totally killing the API calls.

@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Feb 23, 2020
@pmakani pmakani self-assigned this Feb 24, 2020
@pmakani pmakani added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed triage me I really want to be triaged. labels Feb 24, 2020
@pmakani
Copy link
Contributor

pmakani commented Feb 24, 2020

@shevek can you please share your sample code which you have used, version of library and how we can reproduce this issue with steps.

@pmakani pmakani added the needs more info This issue needs more information from the customer to proceed. label Feb 24, 2020
@shevek
Copy link
Author

shevek commented Feb 28, 2020

It looks as if not having the listAllJobs permission is causing BQ server side to omit some critical fields, which might also cause this issue.

@shevek
Copy link
Author

shevek commented Feb 28, 2020

Try submitting a script job as user A, then listing jobs as user B who is not an admin and does not have the listAllJobs permission?

@stephaniewang526
Copy link
Contributor

Closing this ticket as not enough information is provided for us to reproduce the issue. Please refer to https://cloud.google.com/bigquery/docs/managing-jobs for more information listing jobs. We will be adding more Java samples.

@shevek
Copy link
Author

shevek commented Mar 23, 2020

Whu?!

@AzemaBaptiste AzemaBaptiste mentioned this issue Aug 19, 2022
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pmakani pmakani

Labels
api: bigquery Issues related to the googleapis/java-bigquery API. needs more info This issue needs more information from the customer to proceed. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: prevent jobPb.getConfiguration() to throw a NPE AzemaBaptiste/java-bigquery
4 participants
@shevek @stephaniewang526 @pmakani @yoshi-automation