Add support in multiple ports per instance

[zachidan]

Currently an amalgam8 instance has only one endpoint.

Both kubernetes and eureka allows multiple endpoints per service/instance. It sounds reasonable to add support in multiple endpoints to amalgam8.

Adding multiple endpoints requires API changes.

My suggestion is to add a new field of type array, called endpoints, and document the current endpoint field as deprecated. Users can continue to use endpoint, and the new code will continue to set this field (as well as the new endpoints field).

Later, we can decide to drop the endpoint field and stop supporting the old API version.

Current API:
type ServiceInstance struct {
ID string json:"id,omitempty"
ServiceName string json:"service_name,omitempty"
Endpoint *InstanceAddress json:"endpoint,omitempty"
...
}

New API format:
type ServiceInstance struct {
ID string json:"id,omitempty"
ServiceName string json:"service_name,omitempty"
// Endpoint is deprecated. The Endpoints array replaces this field.
Endpoint _InstanceAddress json:"endpoint,omitempty"
Endpoints []_InstanceAddress json:"endpoints,omitempty"
...
}

[rshriram]

Are you referring to multiple ports per instance ? If there are different IP and port pairs for the service instance, there needs to be an additional metadata to identify the endpoint type (we have one already) and an additional field to prioritize which endpoint in the list should the caller prefer.

I am assuming that you simply want to support multiple ports per instance, where endpoint type field is different for each endpoint.
The sidecar can then choose the https endpoint by default.
I would rename the issue as multiple ports per instance.

[zcahana]

@rshriram yes, the intent is to support multiple ports per instance. Using an endpoints array (with each endpoint having a type and a value, as is today) is one way of doing so, with the added flexibility of specifying multiple ip:port pairs for a service instance. If that doesn't make sense, then we can probably just add a ports array (named ports) to the existing endpoint field.

[zcahana]

Adding sidecar tag as this feature should affect the sidecar as well.

Particularly, the sidecar should be aware of multiple endpoints/ports per instance.
Load balancing should be done across service instances (normally, nodes) and not endpoints, whereas routing rules + sensible defaults should handle multiple endpoints per instance.

[frankbu]

I can understand what it means to have an instance that implements multiple endpoints/ports, but I don't understand what it means to have multiple endpoints/ports on a single service, and how it would be exposed in the rules API. I would have thought that if an instance has two endpoints (for example: port 1234 and port 5678) those two endpoints are different services, not both implementing the same service. Wouldn't it be better to handle this by allowing instances to register themselves as implementing two different service (e.g., SERVICE=foo_1234 and SERVICE=foo_5678)?

[zcahana]

@frankbu the discussion here was in the context of multiple endpoints/ports for the a single service. For multiple services, it probably makes more sense to just register separate service instances.

The canonical (and only?) example for multiple ports for a single service is HTTP vs HTTPS.
An example for multiple IPs for the same service instance is when an instance provides the service over multiple network interfaces. Not sure it makes much sense in a containers world, maybe with VMs.