Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migrate BQ Java samples from java-docs-samples to client lib #22

Merged
merged 3 commits into from Dec 12, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 12 additions & 1 deletion pom.xml
Expand Up @@ -246,7 +246,7 @@

<modules>
<module>google-cloud-bigquery</module>
<module>google-cloud-bigquery-bom</module>
<module>google-cloud-bigquery-bom</module>
</modules>

<reporting>
Expand Down Expand Up @@ -341,6 +341,17 @@
<auto-value-annotations.version>${auto-value.version}</auto-value-annotations.version>
</properties>
</profile>

<profile>
<id>samples-java8AndNewer</id>
<activation>
<jdk>[1.8,)</jdk>
</activation>
<modules>
<module>samples</module>
</modules>
</profile>

</profiles>

</project>
48 changes: 48 additions & 0 deletions samples/README.md
@@ -0,0 +1,48 @@
# Getting Started with BigQuery and the Google Java API Client library

<a href="https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/java-docs-samples&page=editor&open_in_editor=README.md&cloudshell_working_dir=bigquery/cloud-client/">
<img alt="Open in Cloud Shell" src ="http://gstatic.com/cloudssh/images/open-btn.png"></a>

[Google's BigQuery Service][BigQuery] features a REST-based API that allows
developers to create applications to run ad-hoc queries on massive datasets.
These sample Java applications demonstrate how to access the BigQuery API using
the [Google Cloud Client Library for Java][google-cloud-java].

[BigQuery]: https://cloud.google.com/bigquery/
[google-cloud-java]: https://github.com/GoogleCloudPlatform/google-cloud-java

## Quickstart

Install [Maven](http://maven.apache.org/).

[Authenticate using a service account](https://cloud.google.com/docs/authentication/getting-started).
Create a service account, download a JSON key file, and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.

Build your project with:

mvn clean package -DskipTests

You can then run a given `ClassName` via:

mvn exec:java -Dexec.mainClass=com.example.bigquery.ClassName \
-DpropertyName=propertyValue \
-Dexec.args="any arguments to the app"

### Creating a new dataset (using the quickstart sample)

mvn exec:java -Dexec.mainClass=com.example.bigquery.QuickstartSample

### Running a query using standard SQL syntax

mvn exec:java -Dexec.mainClass=com.example.bigquery.QuerySample \
-Dexec.args=' \
--query="SELECT corpus FROM `bigquery-public-data.samples.shakespeare` GROUP BY corpus;" \
--runStandardSqlQuery'

### Running the simple app example

To run the example from the [simple app example
documentation](https://cloud.google.com/bigquery/create-simple-app-api):

mvn exec:java -Dexec.mainClass=com.example.bigquery.SimpleApp

76 changes: 76 additions & 0 deletions samples/pom.xml
@@ -0,0 +1,76 @@
<!--
~ Copyright 2019 Google LLC
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.bigquery</groupId>
<artifactId>bigquery-google-cloud-samples</artifactId>
<packaging>jar</packaging>

<!--
The parent pom defines common style checks and testing strategies for our samples.
Removing or replacing it should not affect the execution of the samples in anyway.
-->
<parent>
<groupId>com.google.cloud.samples</groupId>
<artifactId>shared-configuration</artifactId>
<version>1.0.11</version>
</parent>

<properties>
<maven.compiler.target>1.8</maven.compiler.target>
stephaniewang526 marked this conversation as resolved.
Show resolved Hide resolved
<maven.compiler.source>1.8</maven.compiler.source>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<!-- [START bigquery_java_dependencies] -->
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigquery</artifactId>
<version>1.100.0</version>
stephaniewang526 marked this conversation as resolved.
Show resolved Hide resolved
</dependency>
<!-- [END bigquery_java_dependencies] -->
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax</artifactId>
<version>1.49.1</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-core</artifactId>
<version>1.91.3</version>
</dependency>
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<version>0.18.0</version>
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13-beta-3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<version>1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
94 changes: 94 additions & 0 deletions samples/src/main/java/com/example/bigquery/AuthSnippets.java
@@ -0,0 +1,94 @@
/*
* Copyright 2017 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

/**
* Examples for authenticating to Google BigQuery.
*
* <p>See: https://cloud.google.com/bigquery/authentication
*/
public class AuthSnippets {

// [START bigquery_client_default_credentials]
public static void implicit() {
// Instantiate a client. If you don't specify credentials when constructing a client, the
// client library will look for credentials in the environment, such as the
// GOOGLE_APPLICATION_CREDENTIALS environment variable.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

// Use the client.
System.out.println("Datasets:");
for (Dataset dataset : bigquery.listDatasets().iterateAll()) {
System.out.printf("%s%n", dataset.getDatasetId().getDataset());
}
}
// [END bigquery_client_default_credentials]

// [START bigquery_client_json_credentials]
public static void explicit() throws IOException {
// Load credentials from JSON key file. If you can't set the GOOGLE_APPLICATION_CREDENTIALS
// environment variable, you can explicitly load the credentials file to construct the
// credentials.
GoogleCredentials credentials;
File credentialsPath = new File("service_account.json"); // TODO: update to your key path.
try (FileInputStream serviceAccountStream = new FileInputStream(credentialsPath)) {
credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
}

// Instantiate a client.
BigQuery bigquery =
BigQueryOptions.newBuilder().setCredentials(credentials).build().getService();

// Use the client.
System.out.println("Datasets:");
for (Dataset dataset : bigquery.listDatasets().iterateAll()) {
System.out.printf("%s%n", dataset.getDatasetId().getDataset());
}
}
// [END bigquery_client_json_credentials]

public static void main(String... args) throws IOException {
boolean validArgs = args.length == 1;
String sample = "explicit";
if (validArgs) {
sample = args[0];
if (!sample.equals("explicit") && !sample.equals("implicit")) {
validArgs = false;
}
}

if (!validArgs) {
System.err.println("Expected auth type argument: implict|explict");
System.exit(1);
}

if (sample.equals("implicit")) {
implicit();
} else {
explicit();
}
}
}
49 changes: 49 additions & 0 deletions samples/src/main/java/com/example/bigquery/CreateDataset.java
@@ -0,0 +1,49 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_create_dataset]
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import com.google.cloud.bigquery.DatasetInfo;

public class CreateDataset {

public static void runCreateDataset() {
// TODO(developer): Replace these variables before running the sample.
String datasetName = "my-dataset-name";
createDataset(datasetName);
}

public static void createDataset(String datasetName) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
try {
Dataset newDataset = bigquery.create(datasetInfo);
String newDatasetName = newDataset.getDatasetId().getDataset();
System.out.println(newDatasetName + " created successfully");
} catch (BigQueryException e) {
System.out.println("Dataset was not created. \n" + e.toString());
}
}
}
// [END bigquery create_dataset]
48 changes: 48 additions & 0 deletions samples/src/main/java/com/example/bigquery/DeleteDataset.java
@@ -0,0 +1,48 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_delete_dataset]
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQuery.DatasetDeleteOption;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.DatasetId;

public class DeleteDataset {

public static void runDeleteDataset() {
// TODO(developer): Replace these variables before running the sample.\
String projectId = "my-project-id";
String datasetName = "my-dataset-name";
deleteDataset(projectId, datasetName);
}

public static void deleteDataset(String projectId, String datasetName) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

DatasetId datasetId = DatasetId.of(projectId, datasetName);
boolean success = bigquery.delete(datasetId, DatasetDeleteOption.deleteContents());
if (success) {
System.out.println("Dataset deleted successfully");
} else {
System.out.println("Dataset was not found");
}
}
}
// [END bigquery_delete_dataset]
51 changes: 51 additions & 0 deletions samples/src/main/java/com/example/bigquery/ListDatasets.java
@@ -0,0 +1,51 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_list_datasets]
import com.google.api.gax.paging.Page;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQuery.DatasetListOption;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;

public class ListDatasets {

public static void runListDatasets() {
// TODO(developer): Replace these variables before running the sample.
String projectId = "my-project-id";
listDatasets(projectId);
}

public static void listDatasets(String projectId) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

// List datasets in a specified project
try {
Page<Dataset> datasets = bigquery.listDatasets(projectId, DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
System.out.println(dataset.getDatasetId() + " dataset in project listed successfully");
}
} catch (BigQueryException e) {
System.out.println("Project does not contain any datasets \n" + e.toString());
}
}
}
// [END bigquery_list_datasets]