Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configuration option service_prefix for Consul DCS #1543

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

ghost
Copy link

@ghost ghost commented May 12, 2020

To enhance service registration in consul, I'd like to suggest to add a configuration option service_prefix to the consul section of the patroni configuration.

This option allows to prefix the registered service name with an arbitrary string. The need for this option arises as the scope name usually doesn't include a string like patroni- (as this would be redundant in the scope name itself). As a result, the service name registered in consul is very generic and somewhat meaningless / lacking the type-of-service information.

With the change introduced in this PR, a service can be named i.e. patroni-batman or pgdb-myservice instead of just batman or myservice.

The change is fully backwards-compatible.

@ghost ghost changed the title Feature / configuration option service_prefix for Consul DCS Add configuration option service_prefix for Consul DCS May 12, 2020
patroni/dcs/consul.py Outdated Show resolved Hide resolved
Removal of unnecessary private member variable as suggested.

Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com>
@CyberDem0n
Copy link
Collaborator

After looking at it for the second time I've got a question, why is it a prefix, and no a suffix?
The suffix probably better fits into DNS?
Or maybe it makes sense to have both?

@ghost
Copy link
Author

ghost commented May 13, 2020

I decided to go with the prefix approach as it seemed more natural (I'm used to the Debian patroni package, where the systemd service is called e.g. ´patroni@9.6-batman.service´). But in the end, it's personal preference which shouldn't affect usage.

With regards to 'fitting into DNS', I don't think that matters concerning functionality as any service name in consul isn't anything more than a plain string. But that comment led me to another question:

Maybe, the service published by patroni should always be named plain and simple patroni - as, i.e. redis registers a service just named redis - at least according to the consul docs.

The differentiation between different clusters would then happen solely based on consul tags, one of which would be the scope name. I guess this would be the preferred approach, which would require some more changes.

This approach also doesn't come without downsides. Namely, DNS Service Queries against consul support only one single tag, e.g. master.patroni-9-6-batman.service.consul.. When the scope name is also provided as a tag, one cannot use a simple DNS Query to find the master of a specific cluster.

Of course, consul provides 'prepared query templates' stored in the cluster itself to work around that limitation, but that introduces another layer of complexity that I wouldn't want to be required to use for 'simple' use cases.

As a compromise, the following changes could be implemented:

  • Name the registered service patroni by default, but allow the user to specify a custom service name using a service_name configuration option.
  • Always provide an additional tag (beside the role tag already implemented) containing the scope name (or - don't think that provides any value - allow to enable/disable the scope name tag via config option tag_scope)

This way, a user can choose to still manually incorporate the scope name in the service name by just setting service_name to a custom value, alleviating the multi-tag-query issue. Still, by default the most 'consul-like' registration approach is chosen. Also, the 'prefix-vs-suffix' question becomes obsolete by this approach.

@CyberDem0n
Copy link
Collaborator

Ok, it seems that the service_prefix is less invasive and what is more important it keeps backward compatibility.

@ghost
Copy link
Author

ghost commented May 13, 2020

I forgot about backwards compatibilty for a moment.

Considering that, service_name could still be implemented instead of service_prefix, with a default value of the scope name, not changing behaviour at all. This would also be a pretty simple change.

In addition, tag_scope could be added as a boolean parameter to add a consul tag with the scope name. Or the tag could just be always added, as it is just a piece of additional information, not altering behaviour.

If these suggestions are acceptable, I would change this PR to implement the first part and do the tagging thing in a second PR.

@CyberDem0n
Copy link
Collaborator

@awiller42, sounds like a plan. Please put everything into a single PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants