Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nsqd: discover topic/channel paused state on new topic discovery #1199

Open
jehiah opened this issue Oct 16, 2019 · 3 comments · May be fixed by #1274
Open

nsqd: discover topic/channel paused state on new topic discovery #1199

jehiah opened this issue Oct 16, 2019 · 3 comments · May be fixed by #1274

Comments

@jehiah
Copy link
Member

jehiah commented Oct 16, 2019

I'd like to provide an experience report from introducing new nsqd nodes to a cluster while operational activities which involve pausing channels are underway.


Currently nsqd discovers the topic/channel topology from nqlookupd when it receives a new message. This ensures that if you have say 10 channels on a topic, all 10 channels get a copy of the first message.

If a channel is in a paused state when the first message is published to a new nsqd node it's naturally expected that the new node would also end up with channels in an appropriately paused states. That doesn't happen unfortunately because nsqlookupd has no context about the paused state of a channel, and so it can't provide it when queried by nsqd on a new topic creation. (There also isn't technically anything that requires all hosts to be paused or unpaused together, even though that's obviously most common)


Feature Request: nsqd propagate topic/channel pause states to nsqlookupd. On discovery, nsqlookupd can advise if the majority of clients for a topic/channel are paused and if so, nqsd will create a pre-paused topic or channel. If there are no clients for a topic/channel connected to nsqd at the time of topic discovery, nsqd should create a topic and channel un-paused. When nsqd queries multiple nqslookupd if any response indicates paused, the result should be a paused state. i.e. paused is preferred.

Note: nsqlookupd response changes to support this should be backwards compatible.

@mreiferson
Copy link
Member

SGTM

@maplessssy
Copy link

@jehiah @mreiferson any update here? I think i can spend a weekend to finish it.

@maplessssy
Copy link

maplessssy commented Aug 7, 2020

hi, anyone can help to take a look at this?
first step: Command [Register] need support to deliver topic/channel paused state.
see go-nsq pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants