Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add JsonStreamWriter #475

Merged
merged 41 commits into from Aug 7, 2020
Merged

feat: add JsonStreamWriter #475

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
41f5d52
feat: add JsonStreamWriter
allenc3 Jul 10, 2020
db7e05f
Finished basics of JsonStreamWriter
allenc3 Jul 13, 2020
b480897
Working basic JsonStreamWriter
allenc3 Jul 15, 2020
bcdd32f
Attempt to implement schema update
allenc3 Jul 16, 2020
e204c40
Progress on update schema
allenc3 Jul 17, 2020
39bca93
Finished designing of JsonStreamWriter, waiting for JsonToProtoMessag…
allenc3 Jul 21, 2020
f3fd0e4
Update append method to convert json data to proto messages.
allenc3 Jul 21, 2020
3801652
Temp commit, somehow missing protos
allenc3 Jul 22, 2020
225369d
Copied over protos
allenc3 Jul 22, 2020
ecfcd09
Added more tests and finalize JsonStreamWriter design
allenc3 Jul 22, 2020
a54a2d6
ls
allenc3 Jul 22, 2020
5b26c9f
Finished tests
allenc3 Jul 22, 2020
e6d2fbc
Add import for Optional
allenc3 Jul 22, 2020
e68fccb
Removed Optional
allenc3 Jul 22, 2020
3900381
Add multiple append test case
allenc3 Jul 22, 2020
62f9be2
Added test case for schema update exception
allenc3 Jul 23, 2020
f25128b
Adding offset exception test cases
allenc3 Jul 23, 2020
138c94c
Fixing handling exception cases
allenc3 Jul 24, 2020
6d424a6
Add more test case and make sure JsonStreamWriter and StreamWriter ar…
allenc3 Jul 24, 2020
868bd7f
Added IT test
allenc3 Jul 24, 2020
e1d347d
Changed schema update callback to be a runnable and adding ITTests
allenc3 Jul 27, 2020
5e34734
Changed variables to be private and added getter methods for runnable…
allenc3 Jul 27, 2020
11f2fb3
update to IT test
allenc3 Jul 27, 2020
0eb61cd
Finished JsonStreamWriter, need to fix e2e for schema update.
allenc3 Jul 28, 2020
6a574e4
fix minor detail in JsonStreamWriter
allenc3 Jul 28, 2020
6743cc9
More it Tests
allenc3 Jul 28, 2020
de98f32
Fixed according to PR suggestions
allenc3 Jul 29, 2020
50d108f
Trigger rerun
allenc3 Jul 29, 2020
a5ce6e4
Trigger rerun
allenc3 Jul 29, 2020
281645f
Fixed according to suggestions
allenc3 Jul 31, 2020
82bc6bc
modified IT test
allenc3 Jul 31, 2020
da41599
temp commit
allenc3 Jul 31, 2020
63c3dc4
Added e2e schema update test, multi thread append JsonStreamWriterTes…
allenc3 Aug 3, 2020
a0d09be
Remove commentted out code
allenc3 Aug 3, 2020
3c66472
Experimenting on writer schema
allenc3 Aug 5, 2020
c46b164
Fixed according to PR suggestions; dealth with syncrhonization issues…
allenc3 Aug 5, 2020
07de055
Fix null ptr error in StreamWriter
allenc3 Aug 5, 2020
3c8c40e
Added refreshConnection in JsonStreamWriter to let it handle refreshA…
allenc3 Aug 6, 2020
f26ca3b
Made JsonStreamWriterOnSchemaUpdateRunnable a private inner class and…
allenc3 Aug 6, 2020
032c5bb
Rerun
allenc3 Aug 6, 2020
ce3459a
remove import *
allenc3 Aug 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 3 additions & 1 deletion ...igquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/DirectWriter.java
View file
Open in desktop
Expand Up @@ -15,7 +15,9 @@
*/
package com.google.cloud.bigquery.storage.v1alpha2;

import com.google.api.core.*;
import com.google.api.core.ApiFunction;
import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutures;
import com.google.api.gax.grpc.GrpcStatusCode;
import com.google.api.gax.rpc.InvalidArgumentException;
import com.google.cloud.bigquery.storage.v1alpha2.ProtoBufProto.ProtoRows;
Expand Down