BigQuery samples

[gasgallo]

Is there any place where to find complete samples of how to use the BigQuery APIs?
I got BigQueryReadClient to work using the quickstart code, but I'm getting lost in making BigQueryWriteClient to work.

Let's assume I have my schema in JSON format and the data to be written to my BQ table, what I'm doing now:

auto write_client= BigQueryWriteClient(bigquery::MakeBigQueryWriteConnection());
auto stream = write_client.AsyncAppendRows();

google::cloud::bigquery::storage::v1::AppendRowsRequest request;
google::cloud::bigquery::storage::v1::AppendRowsResponse response;

std::string stream_name = "my-stream-name-here";
request.set_write_stream(stream_name);

google::cloud::bigquery::storage::v1::AppendRowsRequest_ProtoData data;  // how to assign my data and schema to AppendRowsRequest_ProtoData?
request.set_allocated_proto_rows(&data);

grpc::WriteOptions options;
stream->Write(request, options);
stream->WritesDone();

Thanks!

[coryan]

google::cloud::bigquery::storage::v1::AppendRowsRequest_ProtoData data; // how to assign my data and schema to AppendRowsRequest_ProtoData?
request.set_allocated_proto_rows(&data);

Probably the easiest thing is to have a protocol buffer with an equivalent schema, such as in this example (it is work in progress, it lacks testing, a good README, etc.):

GoogleCloudPlatform/cpp-samples#211

If you need to do this dynamically, then it will probably get complicated really fast. The API requires uploading the data as serialized protos:

https://github.com/googleapis/googleapis/blob/13d5b3f3f9412f38427c8ad48068f04ad1ee9808/google/cloud/bigquery/storage/v1/storage.proto#L420-L424

So without an actual .proto file, you would need to dynamically create a descriptor for the proto. I have never done this, but the documentation suggests this is a starting point:

https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.descriptor#DescriptorPool

[gasgallo]

It works well, thanks a lot for sharing this POC.