Support service discovery

[minwoox]

We can use Central Dogma servers as the service registry to store the dynamically changed information of services and fetch the information for service discovery.
Instead of defining our own protocol to communicate with the clients, we can use the De facto standard xDS REST and gRPC protocol so that Central Dogma will be integrated well with other discovery clients.

If we use the protocol, Central Dogma will play the role of a control plane so that services push their information to Central Dogma.

How will a service register its information?

• Currently, the XDS protocol does not define the APIs for registering and deregistering so we should define them.
• As XDS protocol supports both REST and gRPC, we need to support both eventually.
  • However, if we use gRPC bidi-streaming, the client would get a notification right away when the source is changed so I prefer to implement gRPC first.

How will the information is stored in Central Dogma?

• The information will be stored in the repository in Central Dogma with the format of Protobuf IDL that is defined by Envoy.
• Because the throughput of write per second of the current repository is too low, we need alternatives;
  • Merging the requests that are received within a certain period of time and committing them at once.
  • Implementing another repository that fastly commits with a lower consistency level.

Note:

• We are not going to remove the information of a service when the connection is disconnected unless the service deregisters itself explicitly via the API.
  • The CSLB client will be in charge of not sending a request to the unhealthy server using the health check.
  • We might want to consider adding a feature that the control plane deregisters a service when the control plane gets a lot of complaints from the CSLB client.