Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adds examples and fixes bugs in spark connector #456

Merged
merged 6 commits into from Jan 13, 2021

Conversation

jiangmichaellll
Copy link
Contributor

This adds a simple read example and fixes the following bugs:

  1. microbatchinputpartition subscriber should initially seek to startoffset instead of endoffset
  2. lambda introduced in previous PRs brings this(PslXXXReader) as part of serialization, this PR detached this in lambda.

@jiangmichaellll jiangmichaellll requested review from a team as code owners January 13, 2021 20:15
@product-auto-label product-auto-label bot added the api: pubsublite Issues related to the googleapis/java-pubsublite API. label Jan 13, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jan 13, 2021
@codecov
Copy link

codecov bot commented Jan 13, 2021

Codecov Report

Merging #456 (768e61a) into master (48a19d7) will decrease coverage by 0.13%.
The diff coverage is 40.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #456      +/-   ##
============================================
- Coverage     71.27%   71.14%   -0.14%     
  Complexity      918      918              
============================================
  Files           168      168              
  Lines          4864     4873       +9     
  Branches        243      244       +1     
============================================
  Hits           3467     3467              
- Misses         1259     1268       +9     
  Partials        138      138              
Impacted Files Coverage Δ Complexity Δ
...le/cloud/pubsublite/spark/PslContinuousReader.java 50.00% <0.00%> (-4.06%) 8.00 <0.00> (ø)
.../pubsublite/spark/PslMicroBatchInputPartition.java 35.00% <16.66%> (-5.00%) 1.00 <0.00> (ø)
...le/cloud/pubsublite/spark/PslMicroBatchReader.java 80.95% <91.66%> (+2.90%) 12.00 <3.00> (ø)
...ubsublite/internal/BlockingPullSubscriberImpl.java 87.93% <0.00%> (-5.18%) 11.00% <0.00%> (-1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48a19d7...768e61a. Read the comment docs.

@jiangmichaellll jiangmichaellll merged commit d1c2a24 into master Jan 13, 2021
@jiangmichaellll jiangmichaellll deleted the jiangmichael-spark-example branch January 13, 2021 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsublite Issues related to the googleapis/java-pubsublite API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants