Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mode -deregister never #646

Open
itrofimow opened this issue Oct 26, 2018 · 1 comment
Open

Support mode -deregister never #646

itrofimow opened this issue Oct 26, 2018 · 1 comment

Comments

@itrofimow
Copy link

It's more like an enhancement request, not an issue.

Request might seem strange, but let me explain further:
I'd like to use Consul as my God-Object, the one who knows everything about my setup/infrastructure/services etc.
Furthermore, i want to use Consul as the source of my monitoring/alerting, however this seems currently impossible with registrator unregistering services when containers are stoped gracefully - there are just no information about them left anywhere in Consul, they are just gone and that's not suitable for me.

The most straightforward case is like this:
If at some point all containers of some service are gracefully stoped (my deployment process involves docker stop at some point, for instance, and things might go wrong) then the only way i can know about this is customers complaining about 502-s.

Yes, i could alert about this exact scenario, when all containers of service are dead - just because this gives information (no service with name example anymore at all) but if just some containers are stoped - i cant know about it without taking my infrastructure configuration from somewhere and complicating things matching it against Consul etc.

The workaround for my case is like this: itrofimow@a772b6e
so when container dies its not unregistered and agent tcp-check fails, thus service becomes critical

I am somewhat concerned - may be consul + consul-registrator are just not the suitable duo for this kind of half-static configuration and i am reinventing the bicycle?

Could you please share your thoughts about this and maybe point me at some non-obvious scenarious i am missing (and yes, i do realize that some services might get stale at consul if hostname/container-name/container-ip changes and features relying on deregistering services wont work anymore)

@itrofimow
Copy link
Author

Seems like i've forgot an important explanation about my deployment process:

  • each service is deployed to some static configuration of nodes in each of DC-s
  • hostname:container-name:service-port are always same and dont change for a pair container/node
  • this involves rebalancing load away from one dc, deployment on it, rebalancing load again, deployment on another dc, rebalancing load on both DC-s
  • deployment is basically:
  1. docker stop
  2. docker rm
  3. docker run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant