Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QQs: support if-empty and if-unused deletion parameters #10543

Open
Zerpet opened this issue Feb 14, 2024 · 2 comments
Open

QQs: support if-empty and if-unused deletion parameters #10543

Zerpet opened this issue Feb 14, 2024 · 2 comments
Assignees
Milestone

Comments

@Zerpet
Copy link
Contributor

Zerpet commented Feb 14, 2024

Is your feature request related to a problem? Please describe.

Classic Queues and Streams support "fail-safe" parameters in queue deletion requests. Quorum Queues could benefit from the same parameters, with some limitations, given the nature of QQs. The intention is to prevent accidental queue deletions when a queue has data and/or consumers.

Observe the following snippet:

Delete queues using rabbitmqctl delete_queue
docker exec -t bunny rabbitmqctl list_queues name type messages consumers
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
name	type	messages	consumers
c1-v1	classic	0	0
st	stream	0	0
qq	quorum	0	0
cq-v2	classic	0	0
~ % docker exec -t bunny rabbitmqctl delete_queue c1-v1 --if-empty --if-unused
Deleting queue 'c1-v1' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages
~ % docker exec -t bunny rabbitmqctl delete_queue cq-v2 --if-empty --if-unused
Deleting queue 'cq-v2' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages
~ % docker exec -t bunny rabbitmqctl delete_queue qq --if-empty --if-unused
Deleting queue 'qq' on vhost '/' if queue is empty and if queue is unused ...
Error:
{:amqp_error, :not_implemented, ~c"cannot delete queue 'qq' in vhost '/'. queue.delete operations with if-unused flag set are not supported by quorum queues", :none}
~ % docker exec -t bunny rabbitmqctl delete_queue st --if-empty --if-unused
Deleting queue 'st' on vhost '/' if queue is empty and if queue is unused ...
Queue was successfully deleted with 0 ready messages

Describe the solution you'd like

Add support for if-empty and if-unused in queue deletion requests. Given the distributed nature of Quorum Queues, our guarantees for if-empty would be as follows: when a queue deletion request arrives, check if there are ready or pending messages. If there are non-zero ready or pending messages, return a failed queue delete response. If there are zero, proceed then with the delete request.

Our guarantees for if-unused would be as follows: when a queue deletion request arrives, check if any member has consumers attached. If there are non-zero consumers, return a failed queue delete response. If there are zero, then proceed with the delete request.

Those guarantees do not prevent the scenario where a message or a consumer arrives after the check has happened, and before the queue deletion is completed. Given the distributed and uncoupled nature of Quorum Queues, I believe this edge case is acceptable. The intention is not to make an atomic "if-empty/unused" queue deletion, but to prevent accidental queue deletions of queues with data or consumers.

Describe alternatives you've considered

No response

Additional context

This came up as part of a contribution to the Topology Operator rabbitmq/messaging-topology-operator#759

Given the lack of support for if-empty/if-unused, the Topology Operator would have to perform a queue.get operation, check if it has messages or consumers, then proceed with the delete request (of the Kubernetes object) accordingly. This workaround has a much larger window where messages or consumers may "sneak in", and it would only benefit Kubernetes users. I believe everyone could benefit from this feature being built-in the core.

@Zerpet Zerpet added this to the 4.0.0 milestone Feb 14, 2024
@michaelklishin michaelklishin changed the title Support if-empty and if-unused delete parameters in Quorum Queues QQs: support if-empty and if-unused deletion parameters Feb 14, 2024
@ikavgo ikavgo self-assigned this Feb 26, 2024
@ikavgo
Copy link
Contributor

ikavgo commented Feb 28, 2024

It appears as if IfUnused and IfEmpty are ignored for streams - https://github.com/rabbitmq/rabbitmq-server/blob/main/deps/rabbit/src/rabbit_stream_queue.erl#L197.

Regarding user experience. The numbers shown have different source depending on the queue type. Classic queues go thru process dictionary for all ch records and use the same mechanics for implementing IfUnused. Quorum queues show consumers via queue_metrics and ets:lookup_element. Stream queue uses yet another way of getting consumer count - it enumerates queue nodes and does rpc.

So regarding guarantees and behaviours I have these question:

  1. does it make sense to let streams respect IfUnused/IfEmpty?
  2. does it make sense to use the same mechanics from info/2 that used to generate values displayed to user to implement IfUnused/IfEmpty for streams and quorum queues?
  3. Quorum queues go to metrics for consumer counts and streams enumerate nodes and also doing ets calls. Shouldn't they look the same?

@Zerpet
Copy link
Contributor Author

Zerpet commented Feb 29, 2024

does it make sense to let streams respect IfUnused/IfEmpty?

I don't think so, specially if they are just ignored at the moment. Streams are expected to have data all the time when, at least, one message is published. Perhaps it could support if-unused, I don't have a strong opinion on that regard. My intention with this issue was to focus primarily on Quorum Queues.

I can't comment on your other questions since I have little-to-none context on those areas 🙃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants