Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BigQuery StorageWrite: JsonToProtoMessage timestamp error #1515

Closed
caseyduquettesc opened this issue Feb 6, 2022 · 0 comments · Fixed by #1574 or #1567
Closed

BigQuery StorageWrite: JsonToProtoMessage timestamp error #1515

caseyduquettesc opened this issue Feb 6, 2022 · 0 comments · Fixed by #1574 or #1567
Assignees
@stephaniewang526
Labels
api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@caseyduquettesc
Copy link

caseyduquettesc commented Feb 6, 2022 •
edited

Environment details

  1. BigQuery StorageWrite
  2. OS type and version: MacOS 11.6.3
  3. Java version: 11
  4. version(s): 2.8.4

Steps to reproduce

  1. Create a bigquery table with only one field of type TIMESTAMP and with name "d"
  2. Use JsonStreamWriter with this JSON Array [{"d":"2022-02-06 07:24:47.84"}]
  3. run

Stack trace

java.lang.IllegalArgumentException: JSONObject does not have a int64 field at root.d.
	at com.google.cloud.bigquery.storage.v1.JsonToProtoMessage.fillField(JsonToProtoMessage.java:351)
	at com.google.cloud.bigquery.storage.v1.JsonToProtoMessage.convertJsonToProtoMessageImpl(JsonToProtoMessage.java:176)
	at com.google.cloud.bigquery.storage.v1.JsonToProtoMessage.convertJsonToProtoMessage(JsonToProtoMessage.java:115)
	at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.append(JsonStreamWriter.java:140)
	at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.append(JsonStreamWriter.java:99)
	at persistence.BigqueryStreamingClient.streamJsonArray(BigqueryStreamingClient.java:72)

External references such as API reference guides

  • ?

Any additional information below

This is pretty similar to #1330 / #1345, but happens with TIMESTAMP columns. I was wondering if the same fix could be applied? If a string is provided for a TIMESTAMP column, can you convert it? The legacy BQ insert api did and would just make it easer to migrate. Thanks!
@product-auto-label product-auto-label bot added the api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. label Feb 6, 2022
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Feb 7, 2022
@stephaniewang526 stephaniewang526 removed the triage me I really want to be triaged. label Feb 7, 2022
@stephaniewang526 stephaniewang526 self-assigned this Feb 7, 2022
@stephaniewang526 stephaniewang526 added type: question Request for information or clarification. Not an issue. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed type: question Request for information or clarification. Not an issue. labels Feb 7, 2022
stephaniewang526 added a commit to stephaniewang526/java-bigquerystorage that referenced this issue Mar 14, 2022
@stephaniewang526
feat: add JsonToProtoMessage support for TIMESTAMP
f0662da 
Fixes googleapis#1515
@stephaniewang526 stephaniewang526 mentioned this issue Mar 14, 2022
Merged
stephaniewang526 added a commit to stephaniewang526/java-bigquerystorage that referenced this issue Mar 14, 2022
@stephaniewang526
feat: add JsonToProtoMessage support for TIMESTAMP
1e1bc34 
Fixes googleapis#1515

fix unit test

fix lint

update IT

🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
gcf-merge-on-green bot pushed a commit that referenced this issue Mar 14, 2022
@stephaniewang526
feat: add JsonToProtoMessage support for TIMESTAMP (#1574)
6412fb2 
Fixes #1515
@release-please release-please bot mentioned this issue Mar 14, 2022
Merged
gcf-merge-on-green bot pushed a commit that referenced this issue Mar 14, 2022
@release-please
chore(main): release 2.11.0 (#1567)
fc2b2de 
🤖 I have created a release *beep* *boop*
---


## [2.11.0](v2.10.1...v2.11.0) (2022-03-14)


### Features

* add JsonToProtoMessage support for TIMESTAMP ([#1574](#1574)) ([6412fb2](6412fb2)), closes [#1515](#1515)


### Bug Fixes

* a possible race condition that we used table schema out of the lock. ([#1575](#1575)) ([b587638](b587638))


### Dependencies

* update dependency com.google.cloud:google-cloud-bigquery to v2.9.1 ([#1562](#1562)) ([7efdbf0](7efdbf0))
* update dependency com.google.cloud:google-cloud-bigquery to v2.9.3 ([#1570](#1570)) ([3d281d2](3d281d2))
* update dependency com.google.cloud:google-cloud-bigquery to v2.9.4 ([#1572](#1572)) ([d3935cd](d3935cd))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
@dark0dave dark0dave mentioned this issue Mar 15, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stephaniewang526 stephaniewang526

Labels
api: bigquerystorage Issues related to the googleapis/java-bigquerystorage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
3 participants
@stephaniewang526 @caseyduquettesc @yoshi-automation