Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide single threaded multiplexing of connections and subscription groups operations #1857

Open
mensfeld opened this issue Jan 18, 2024 · 1 comment
Assignees

Comments

@mensfeld
Copy link
Member

Currently we use threads to operate all the subscriptions (listeners). While this is ok, we could take advantage of the event-based API from librdkafka to be able to do almost the same from a single thread. We would have to however ensure that:

  1. None of listeners leak into the managing loop
  2. They can be managed as they are
  3. Their event bus operates independently (as it does now) for events publication
  4. They are blocked only in their respective SG when processing
@mensfeld mensfeld self-assigned this Jan 18, 2024
@mensfeld
Copy link
Member Author

Every single waiting flow could yield from a fiber to the managing thread.

@mensfeld mensfeld changed the title Provide single threaded multiplexing of connections and consumer group operations Provide single threaded multiplexing of connections and subscription groups operations Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant