Skip to content

Commit

Permalink
feat: migrate BQ Java samples from java-docs-samples to client lib (#22)
Browse files Browse the repository at this point in the history
* feat: migrate BQ Java samples from java-docs-samples to client lib

* Move sample module include into profile

* Fix dependency issues
  • Loading branch information
stephaniewang526 committed Dec 12, 2019
1 parent 841d075 commit f0bdc4d
Show file tree
Hide file tree
Showing 21 changed files with 1,173 additions and 1 deletion.
13 changes: 12 additions & 1 deletion pom.xml
Expand Up @@ -246,7 +246,7 @@

<modules>
<module>google-cloud-bigquery</module>
<module>google-cloud-bigquery-bom</module>
<module>google-cloud-bigquery-bom</module>
</modules>

<reporting>
Expand Down Expand Up @@ -341,6 +341,17 @@
<auto-value-annotations.version>${auto-value.version}</auto-value-annotations.version>
</properties>
</profile>

<profile>
<id>samples-java8AndNewer</id>
<activation>
<jdk>[1.8,)</jdk>
</activation>
<modules>
<module>samples</module>
</modules>
</profile>

</profiles>

</project>
48 changes: 48 additions & 0 deletions samples/README.md
@@ -0,0 +1,48 @@
# Getting Started with BigQuery and the Google Java API Client library

<a href="https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/java-docs-samples&page=editor&open_in_editor=README.md&cloudshell_working_dir=bigquery/cloud-client/">
<img alt="Open in Cloud Shell" src ="http://gstatic.com/cloudssh/images/open-btn.png"></a>

[Google's BigQuery Service][BigQuery] features a REST-based API that allows
developers to create applications to run ad-hoc queries on massive datasets.
These sample Java applications demonstrate how to access the BigQuery API using
the [Google Cloud Client Library for Java][google-cloud-java].

[BigQuery]: https://cloud.google.com/bigquery/
[google-cloud-java]: https://github.com/GoogleCloudPlatform/google-cloud-java

## Quickstart

Install [Maven](http://maven.apache.org/).

[Authenticate using a service account](https://cloud.google.com/docs/authentication/getting-started).
Create a service account, download a JSON key file, and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.

Build your project with:

mvn clean package -DskipTests

You can then run a given `ClassName` via:

mvn exec:java -Dexec.mainClass=com.example.bigquery.ClassName \
-DpropertyName=propertyValue \
-Dexec.args="any arguments to the app"

### Creating a new dataset (using the quickstart sample)

mvn exec:java -Dexec.mainClass=com.example.bigquery.QuickstartSample

### Running a query using standard SQL syntax

mvn exec:java -Dexec.mainClass=com.example.bigquery.QuerySample \
-Dexec.args=' \
--query="SELECT corpus FROM `bigquery-public-data.samples.shakespeare` GROUP BY corpus;" \
--runStandardSqlQuery'

### Running the simple app example

To run the example from the [simple app example
documentation](https://cloud.google.com/bigquery/create-simple-app-api):

mvn exec:java -Dexec.mainClass=com.example.bigquery.SimpleApp

76 changes: 76 additions & 0 deletions samples/pom.xml
@@ -0,0 +1,76 @@
<!--
~ Copyright 2019 Google LLC
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.bigquery</groupId>
<artifactId>bigquery-google-cloud-samples</artifactId>
<packaging>jar</packaging>

<!--
The parent pom defines common style checks and testing strategies for our samples.
Removing or replacing it should not affect the execution of the samples in anyway.
-->
<parent>
<groupId>com.google.cloud.samples</groupId>
<artifactId>shared-configuration</artifactId>
<version>1.0.11</version>
</parent>

<properties>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<!-- [START bigquery_java_dependencies] -->
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigquery</artifactId>
<version>1.100.0</version>
</dependency>
<!-- [END bigquery_java_dependencies] -->
<dependency>
<groupId>com.google.api</groupId>
<artifactId>gax</artifactId>
<version>1.49.1</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-core</artifactId>
<version>1.91.3</version>
</dependency>
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<version>0.18.0</version>
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13-beta-3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<version>1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
94 changes: 94 additions & 0 deletions samples/src/main/java/com/example/bigquery/AuthSnippets.java
@@ -0,0 +1,94 @@
/*
* Copyright 2017 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

/**
* Examples for authenticating to Google BigQuery.
*
* <p>See: https://cloud.google.com/bigquery/authentication
*/
public class AuthSnippets {

// [START bigquery_client_default_credentials]
public static void implicit() {
// Instantiate a client. If you don't specify credentials when constructing a client, the
// client library will look for credentials in the environment, such as the
// GOOGLE_APPLICATION_CREDENTIALS environment variable.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

// Use the client.
System.out.println("Datasets:");
for (Dataset dataset : bigquery.listDatasets().iterateAll()) {
System.out.printf("%s%n", dataset.getDatasetId().getDataset());
}
}
// [END bigquery_client_default_credentials]

// [START bigquery_client_json_credentials]
public static void explicit() throws IOException {
// Load credentials from JSON key file. If you can't set the GOOGLE_APPLICATION_CREDENTIALS
// environment variable, you can explicitly load the credentials file to construct the
// credentials.
GoogleCredentials credentials;
File credentialsPath = new File("service_account.json"); // TODO: update to your key path.
try (FileInputStream serviceAccountStream = new FileInputStream(credentialsPath)) {
credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
}

// Instantiate a client.
BigQuery bigquery =
BigQueryOptions.newBuilder().setCredentials(credentials).build().getService();

// Use the client.
System.out.println("Datasets:");
for (Dataset dataset : bigquery.listDatasets().iterateAll()) {
System.out.printf("%s%n", dataset.getDatasetId().getDataset());
}
}
// [END bigquery_client_json_credentials]

public static void main(String... args) throws IOException {
boolean validArgs = args.length == 1;
String sample = "explicit";
if (validArgs) {
sample = args[0];
if (!sample.equals("explicit") && !sample.equals("implicit")) {
validArgs = false;
}
}

if (!validArgs) {
System.err.println("Expected auth type argument: implict|explict");
System.exit(1);
}

if (sample.equals("implicit")) {
implicit();
} else {
explicit();
}
}
}
49 changes: 49 additions & 0 deletions samples/src/main/java/com/example/bigquery/CreateDataset.java
@@ -0,0 +1,49 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_create_dataset]
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import com.google.cloud.bigquery.DatasetInfo;

public class CreateDataset {

public static void runCreateDataset() {
// TODO(developer): Replace these variables before running the sample.
String datasetName = "my-dataset-name";
createDataset(datasetName);
}

public static void createDataset(String datasetName) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
try {
Dataset newDataset = bigquery.create(datasetInfo);
String newDatasetName = newDataset.getDatasetId().getDataset();
System.out.println(newDatasetName + " created successfully");
} catch (BigQueryException e) {
System.out.println("Dataset was not created. \n" + e.toString());
}
}
}
// [END bigquery create_dataset]
48 changes: 48 additions & 0 deletions samples/src/main/java/com/example/bigquery/DeleteDataset.java
@@ -0,0 +1,48 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_delete_dataset]
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQuery.DatasetDeleteOption;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.DatasetId;

public class DeleteDataset {

public static void runDeleteDataset() {
// TODO(developer): Replace these variables before running the sample.\
String projectId = "my-project-id";
String datasetName = "my-dataset-name";
deleteDataset(projectId, datasetName);
}

public static void deleteDataset(String projectId, String datasetName) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

DatasetId datasetId = DatasetId.of(projectId, datasetName);
boolean success = bigquery.delete(datasetId, DatasetDeleteOption.deleteContents());
if (success) {
System.out.println("Dataset deleted successfully");
} else {
System.out.println("Dataset was not found");
}
}
}
// [END bigquery_delete_dataset]
51 changes: 51 additions & 0 deletions samples/src/main/java/com/example/bigquery/ListDatasets.java
@@ -0,0 +1,51 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example.bigquery;

// [START bigquery_list_datasets]
import com.google.api.gax.paging.Page;
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQuery.DatasetListOption;
import com.google.cloud.bigquery.BigQueryException;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;

public class ListDatasets {

public static void runListDatasets() {
// TODO(developer): Replace these variables before running the sample.
String projectId = "my-project-id";
listDatasets(projectId);
}

public static void listDatasets(String projectId) {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

// List datasets in a specified project
try {
Page<Dataset> datasets = bigquery.listDatasets(projectId, DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
System.out.println(dataset.getDatasetId() + " dataset in project listed successfully");
}
} catch (BigQueryException e) {
System.out.println("Project does not contain any datasets \n" + e.toString());
}
}
}
// [END bigquery_list_datasets]

0 comments on commit f0bdc4d

Please sign in to comment.