feat: Add the PartitionCountWatchingPublisher #387

palmere-google · 2020-11-22T18:08:07Z

This publisher watches the partition count and adds partitions when the topic's partition count increases.

...loud-pubsublite/src/main/java/com/google/cloud/pubsublite/internal/DefaultRoutingPolicy.java

dpcollins-google · 2020-11-24T15:09:26Z

...src/main/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisher.java

+      publishers.values().forEach(ApiService::stopAsync);
+      publishers.values().forEach(ApiService::awaitTerminated);
+    }
+  }


Isn't this RoutingPublisher, almost exactly? Can't you just use RoutingPublisher for this?

So I spent a while trying to do that. The awkward bit is that RoutingPublisher is a ProxyApiService which contains all of its partition level publishers.

This makes it challenging to install an new routing publisher when the number of partitions changes, since it's not currently possible to add started services to a ProxyApiService, and it's not possible to remove services either.

...src/main/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisher.java

...n/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisherBuilder.java

...test/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisherTest.java

codecov · 2020-11-24T20:14:11Z

Codecov Report

Merging #387 (ab797b3) into master (6e1f755) will increase coverage by 0.44%.
The diff coverage is 89.28%.

@@             Coverage Diff              @@
##             master     #387      +/-   ##
============================================
+ Coverage     72.39%   72.83%   +0.44%     
- Complexity      783      801      +18     
============================================
  Files           148      150       +2     
  Lines          4191     4300     +109     
  Branches        204      215      +11     
============================================
+ Hits           3034     3132      +98     
- Misses         1042     1050       +8     
- Partials        115      118       +3

Impacted Files	Coverage Δ	Complexity Δ
.../wire/PartitionCountWatchingPublisherSettings.java	`33.33% <33.33%> (ø)`	`2.00 <2.00> (?)`
...loud/pubsublite/internal/DefaultRoutingPolicy.java	`52.94% <75.00%> (+2.94%)`	`2.00 <0.00> (ø)`
...internal/wire/PartitionCountWatchingPublisher.java	`98.92% <98.92%> (ø)`	`17.00 <17.00> (?)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e1f755...ab797b3. Read the comment docs.

...src/main/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisher.java

.../java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisherSettings.java

...test/java/com/google/cloud/pubsublite/internal/wire/PartitionCountWatchingPublisherTest.java

...ud-pubsublite/src/main/java/com/google/cloud/pubsublite/internal/testing/FakeApiService.java

palmere-google requested a review from dpcollins-google November 22, 2020 18:08

palmere-google requested a review from a team as a code owner November 22, 2020 18:08

google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Nov 22, 2020

product-auto-label bot added the api: pubsublite Issues related to the googleapis/java-pubsublite API. label Nov 22, 2020