Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Favor replacing backpressure detection heuristic with write buffer threshold callbacks #1043

Open
vitaly-krugl opened this issue May 10, 2018 · 1 comment
Assignees
Milestone

Comments

@vitaly-krugl
Copy link
Member

vitaly-krugl commented May 10, 2018

Pika's current backpressure detection mechanism is meant as a heuristic to detect when the broker is blocking the connection. RabbitMQ has for some time supported Connection.Blocked/Unblocked notifications (also supported by Pika via add_on_connection_blocked_callback() and add_on_connection_unblocked_callback()), so at least with RabbitMQ, the latter is preferred.

Backpressure detection in Pika (Connection._detect_backpressure()) is presently slated for deprecation since (at least) RabbitMQ now makes backpressure explicit through Connection.Blocked and Connection.Unblocked notifications.

What may be more useful to users of Pika's non-blocking connections for managing memory usage is a simpler mechanism that notifies them when the connection's write buffer size exceeds a certain user-configured high-water mark (so they may suspend publishing to avoid overwhelming the app's memory) and when it goes below a user-configured low-water mark (so they may resume publishing). This concept is akin to asyncio's WriteTransport.set_write_buffer_limits().

@vitaly-krugl
Copy link
Member Author

vitaly-krugl commented May 10, 2018

cc @lukebakken, @michaelklishin

@vitaly-krugl vitaly-krugl changed the title Favor write buffer threshold callbacks over backpressure detection callbacks Favor adding write buffer threshold callbacks over backpressure detection callbacks May 10, 2018
@vitaly-krugl vitaly-krugl changed the title Favor adding write buffer threshold callbacks over backpressure detection callbacks Favor replacing backpressure detection heuristic with write buffer threshold callbacks May 10, 2018
@lukebakken lukebakken added this to the 1.3.0 milestone May 24, 2022
@lukebakken lukebakken self-assigned this May 24, 2022
@lukebakken lukebakken modified the milestones: 1.3.0, 2.0.0 Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants