Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQueryStorage: Connection API uses GOOGLE_APPLICATION_CREDENTIALS instead of credentials configured on instance. #2531

Open
jonathanswenson opened this issue Feb 15, 2023 · 2 comments
Labels
api: bigquery Issues related to the googleapis/java-bigquery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@jonathanswenson
Copy link
Contributor

jonathanswenson commented Feb 15, 2023

Thanks for stopping by to let us know something could be better!

Environment details

  1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
    General, Core, and Other are also allowed as types
  2. OS type and version: Mac Ventura / Linux Ubuntu /
  3. Java version: 17 adoptium
  4. version(s): 2.22.0 2.19.1

Steps to reproduce

  1. create client with non-default credentials.
  2. run query using that takes longer than 10 seconds so that it falls out of the fast query pathway
  3. 💥

Code:

// don't have the GOOGLE_APPLICATION_CREDENTIALS env variable set.
BigQueryCredentials credentials = ServiceAccountCredentials.fromStream(inStream)

BigQueryOptions bigQueryOptions = BigQueryOptions.newBuilder()
    .setProjectId(projectId)
    .setCredentials(credentials)
            .setLocation(region)
            .build()
BigQuery bigquery = bigQueryOptions.getService()
ConneptionSettings connectionSettings = ConnectionSettings.newBuilder()
       .setUseReadAPI(false)
       .setRequestTimeout(10L)
       .setMaxResults(100L)
       .setUseQueryCache(true)
       .build();
Connection connection = bigquery.createConnection(connectionSettings);
String selectQuery = "<query that takes a little while and moves past the initial jobs.query timeout";
ListenableFuture<ExecuteSelectResponse> executeSelectFuture = connection.executeSelectAsync(selectQuery, ...);

Stack trace

The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

External references such as API reference guides

If the highThroughPut read path is hit, a BigQueryReadClient is created using the default BigQueryReadClient.create() method. here.

Because no credentials are passed, this looks for the GOOGLE_APPLICATION_CREDENTIALS instead of passing along the credentials that are configured for the bigquery client being used by the Connection.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/java-bigquery API. label Feb 15, 2023
@jonathanswenson
Copy link
Contributor Author

Related to #2508 -- isn't really possible to run queries that run over 10 seconds unless you use the GOOGLE_APPLICATION_CREDENTIALS

@Neenu1995 Neenu1995 added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Feb 24, 2023
@prash-mi
Copy link
Contributor

Related to #2234

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/java-bigquery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

3 participants