You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Classic Queues and Streams support "fail-safe" parameters in queue deletion requests. Quorum Queues could benefit from the same parameters, with some limitations, given the nature of QQs. The intention is to prevent accidental queue deletions when a queue has data and/or consumers.
Observe the following snippet:
Delete queues using rabbitmqctl delete_queue
docker exec -t bunny rabbitmqctl list_queues name type messages consumers
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name type messages consumers
c1-v1 classic 0 0
st stream 0 0
qq quorum 0 0
cq-v2 classic 0 0
~ % docker exec -t bunny rabbitmqctl delete_queue c1-v1 --if-empty --if-unused
Deleting queue 'c1-v1' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages
~ % docker exec -t bunny rabbitmqctl delete_queue cq-v2 --if-empty --if-unused
Deleting queue 'cq-v2' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages
~ % docker exec -t bunny rabbitmqctl delete_queue qq --if-empty --if-unused
Deleting queue 'qq' on vhost '/' if queue is empty and if queue is unused ...
Error:
{:amqp_error, :not_implemented, ~c"cannot delete queue 'qq' in vhost '/'. queue.delete operations with if-unused flag set are not supported by quorum queues", :none}
~ % docker exec -t bunny rabbitmqctl delete_queue st --if-empty --if-unused
Deleting queue 'st' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages
Describe the solution you'd like
Add support for if-empty and if-unused in queue deletion requests. Given the distributed nature of Quorum Queues, our guarantees for if-empty would be as follows: when a queue deletion request arrives, check if there are ready or pending messages. If there are non-zero ready or pending messages, return a failed queue delete response. If there are zero, proceed then with the delete request.
Our guarantees for if-unused would be as follows: when a queue deletion request arrives, check if any member has consumers attached. If there are non-zero consumers, return a failed queue delete response. If there are zero, then proceed with the delete request.
Those guarantees do not prevent the scenario where a message or a consumer arrives after the check has happened, and before the queue deletion is completed. Given the distributed and uncoupled nature of Quorum Queues, I believe this edge case is acceptable. The intention is not to make an atomic "if-empty/unused" queue deletion, but to prevent accidental queue deletions of queues with data or consumers.
Given the lack of support for if-empty/if-unused, the Topology Operator would have to perform a queue.get operation, check if it has messages or consumers, then proceed with the delete request (of the Kubernetes object) accordingly. This workaround has a much larger window where messages or consumers may "sneak in", and it would only benefit Kubernetes users. I believe everyone could benefit from this feature being built-in the core.
The text was updated successfully, but these errors were encountered:
michaelklishin
changed the title
Support if-empty and if-unused delete parameters in Quorum Queues
QQs: support if-empty and if-unused deletion parameters
Feb 14, 2024
Regarding user experience. The numbers shown have different source depending on the queue type. Classic queues go thru process dictionary for all ch records and use the same mechanics for implementing IfUnused. Quorum queues show consumers via queue_metrics and ets:lookup_element. Stream queue uses yet another way of getting consumer count - it enumerates queue nodes and does rpc.
So regarding guarantees and behaviours I have these question:
does it make sense to let streams respect IfUnused/IfEmpty?
does it make sense to use the same mechanics from info/2 that used to generate values displayed to user to implement IfUnused/IfEmpty for streams and quorum queues?
Quorum queues go to metrics for consumer counts and streams enumerate nodes and also doing ets calls. Shouldn't they look the same?
does it make sense to let streams respect IfUnused/IfEmpty?
I don't think so, specially if they are just ignored at the moment. Streams are expected to have data all the time when, at least, one message is published. Perhaps it could support if-unused, I don't have a strong opinion on that regard. My intention with this issue was to focus primarily on Quorum Queues.
I can't comment on your other questions since I have little-to-none context on those areas 🙃
Is your feature request related to a problem? Please describe.
Classic Queues and Streams support "fail-safe" parameters in queue deletion requests. Quorum Queues could benefit from the same parameters, with some limitations, given the nature of QQs. The intention is to prevent accidental queue deletions when a queue has data and/or consumers.
Observe the following snippet:
Delete queues using
rabbitmqctl delete_queue
Describe the solution you'd like
Add support for
if-empty
andif-unused
in queue deletion requests. Given the distributed nature of Quorum Queues, our guarantees forif-empty
would be as follows: when a queue deletion request arrives, check if there are ready or pending messages. If there are non-zero ready or pending messages, return a failed queue delete response. If there are zero, proceed then with the delete request.Our guarantees for
if-unused
would be as follows: when a queue deletion request arrives, check if any member has consumers attached. If there are non-zero consumers, return a failed queue delete response. If there are zero, then proceed with the delete request.Those guarantees do not prevent the scenario where a message or a consumer arrives after the check has happened, and before the queue deletion is completed. Given the distributed and uncoupled nature of Quorum Queues, I believe this edge case is acceptable. The intention is not to make an atomic "if-empty/unused" queue deletion, but to prevent accidental queue deletions of queues with data or consumers.
Describe alternatives you've considered
No response
Additional context
This came up as part of a contribution to the Topology Operator rabbitmq/messaging-topology-operator#759
Given the lack of support for
if-empty
/if-unused
, the Topology Operator would have to perform aqueue.get
operation, check if it has messages or consumers, then proceed with the delete request (of the Kubernetes object) accordingly. This workaround has a much larger window where messages or consumers may "sneak in", and it would only benefit Kubernetes users. I believe everyone could benefit from this feature being built-in the core.The text was updated successfully, but these errors were encountered: