Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: shutdown event loop if publisher fails to start and set exception on result future #124

Merged
merged 19 commits into from Jun 2, 2021

Conversation

hannahrogers-google
Copy link
Contributor

When publishing to a topic that does not exist, the publisher fails to start because the call to get topic partitions fails with 'Not found'. This change does the following:

  1. Shuts down the event loop for the single publisher on failure
  2. Returns a failed future, instead of throwing the 'not found' error

@hannahrogers-google hannahrogers-google requested a review from a team as a code owner April 26, 2021 22:13
@product-auto-label product-auto-label bot added the api: pubsublite Issues related to the googleapis/python-pubsublite API. label Apr 26, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Apr 26, 2021
try:
self._managed_loop.submit(self._underlying.__aenter__()).result()
except GoogleAPICallError:
self._managed_loop.__exit__(None, None, None)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't actually do this. This will result in the managed loop being closed twice when a context manager is used. You shouldn't try to catch this exception- the caller still has to call exit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, the problem is that we are not calling exit on SinglePublisherImpl from the MultiplexedPublisherClient.

The MultiplexedPublisherClient attempts to get_or_create a SinglePublisherImpl here:

The ClientMultiplexer will attempt to create and start the client here:

But, in the Not Found topic case, the client fails on enter because the call to getTopicPartitions fails. So, this raises an exception and the new publisher client is never assigned to live_clients[key].

When exit is eventually called on the ClientMultiplexer, we will never call exit on the 'not found' client because it was never assigned to live_clients[key]:

Therefore, we will never run exit on the managed_loop. I will re-work this PR to address the real issue.

@dpcollins-google dpcollins-google added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 13, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 13, 2021
@meredithslota
Copy link

@dpcollins-google Is this ok to merge? It seems like it but just want to check.

@anguillanneuf anguillanneuf merged commit c2c2b00 into googleapis:master Jun 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsublite Issues related to the googleapis/python-pubsublite API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants