feat: Add support for increasing partitions in python #74
Conversation
product-auto-label
bot
added
the
api: pubsublite
google-cla
bot
added
the
cla: yes
palmere-google
force-pushed
the
palmere-increase
branch
from
ca3b545
to
dcdfb85
Compare
palmere-google
force-pushed
the
palmere-increase
branch
from
dcdfb85
to
fdddd97
Compare
google/cloud/pubsublite/internal/wire/partition_count_watching_publisher.py
Outdated
Show resolved
Hide resolved
google/cloud/pubsublite/internal/wire/partition_count_watching_publisher.py
Outdated
Show resolved
Hide resolved
google/cloud/pubsublite/internal/wire/partition_count_watching_publisher.py
Outdated
Show resolved
Hide resolved
| def set_box(): | ||
| box.val = PartitionCountWatcherImpl(mock_admin, topic, 0.001) | ||
|
|
||
| # Initialize publisher on another thread with a different event loop. |
There was a problem hiding this comment.
Any asyncio class cannot be accessed from a different event loop, or it is incorrect.
There was a problem hiding this comment.
Okay, before I didn't understand this was necessarily constructed in the same event loop as the publisher.
I still kind of think the property that everything works even if you call aenter from a different thread than you constructed the watcher is nice. I'm happy to remove it though if you think that would be better
There was a problem hiding this comment.
Can you add a utility to the testing helpers that does exactly this? I think this is a good idea, and we should do this more places. I.e. a "runOnThread" function that takes a Callable[[], T] and returns a T
palmere-google
force-pushed
the
palmere-increase
branch
3 times, most recently
from
4528a93
to
43ea68d
Compare
palmere-google
force-pushed
the
palmere-increase
branch
from
43ea68d
to
1e8e5a5
Compare
| Partition(index): self._publisher_factory(Partition(index)) | ||
| for index in range(current_count, partition_count) | ||
| } | ||
| await asyncio.gather(*[p.__aenter__() for p in new_publishers.values()]) |
There was a problem hiding this comment.
run nox from the root directory before merging this.
.gitignore
Outdated
| @@ -19,6 +19,7 @@ develop-eggs | |||
| .installed.cfg | |||
| lib | |||
| lib64 | |||
| venv | |||
There was a problem hiding this comment.
put this down with 'env/' and add the slash.
dpcollins-google
changed the title
palmere-google
force-pushed
the
palmere-increase
branch
from
ed857e5
to
9ef995d
Compare
palmere-google
force-pushed
the
palmere-increase
branch
from
9ef995d
to
4440ad3
Compare
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 馃