Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feature: Added JsonStreamWriter, which is built on top of a StreamWri…
…ter and writes Json data by first converting to protobuf messages then calling StreamWriter's append (#437) * Added JsonStreamWriter * Finished basics of JsonStreamWriter * Working basic JsonStreamWriter * Attempt to implement schema update * Progress on update schema * Finished designing of JsonStreamWriter, waiting for JsonToProtoMessage to be approved. * Update append method to convert json data to proto messages. * Temp commit, somehow missing protos * Copied over protos * Added more tests and finalize JsonStreamWriter design * ls * Finished tests * Add import for Optional * Removed Optional * Add multiple append test case * Added test case for schema update exception * Adding offset exception test cases * Fixing handling exception cases * Add more test case and make sure JsonStreamWriter and StreamWriter are correct * Added IT test * Changed schema update callback to be a runnable and adding ITTests * Changed variables to be private and added getter methods for runnable, changing IT tests now * update to IT test * Finished JsonStreamWriter, need to fix e2e for schema update. * fix minor detail in JsonStreamWriter * More it Tests * Fixed according to PR suggestions * Trigger rerun * Trigger rerun * Fixed according to suggestions * modified IT test * temp commit * Added e2e schema update test, multi thread append JsonStreamWriterTest, and fixed refreshAppend in StreamWriter * Remove commentted out code * Experimenting on writer schema * Fixed according to PR suggestions; dealth with syncrhonization issues by letting StreamWriter handle sending first writer schema, added an additional lock between append and refreshAppend for StreamWriter * Fix null ptr error in StreamWriter * Added refreshConnection in JsonStreamWriter to let it handle refreshAppend, flush, and setDescriptor. * Made JsonStreamWriterOnSchemaUpdateRunnable a private inner class and made refreshConnection not public * Rerun * remove import *
- Loading branch information