Consider supporting Docker Linux containers running in Azure Service Fabric

[MedAnd]

Current implementation uses Consul but would you consider supporting Docker Linux containers running in Azure Service Fabric?

[jwreagor]

Thanks for the suggestion.

I'm not yet familiar enough with the platform or the benefits of running ContainerPilot there. A quick look at their documentation leads me to believe that much of ContainerPilot's underlying health and service discovery would need to be reworked to accommodate their APIs. We're happy with how Consul and HashiCorp tools work across cloud providers today so I don't see Joyent sponsoring this integration for the moment.

But I'd gladly help anyone working on this integration themselves if they decided to fork ContainerPilot. That kind of growth would always lead to more considerations of upstreaming work.

What makes Azure interesting for you? Where do you feel ContainerPilot might benefit on that platform?

[MedAnd]

Hi @cheapRoc, one of the appealing benefits of Azure Service Fabric is that it works on-prem, in Azure, AWS (Joynet?) etc. Given the industry shift to containers I would like to deploy Docker Linux containers in Azure Service Fabric. I'm thinking ContainerPilot would allow me to compose and control container clusters better, inter-dependencies which are not strictly possible with Azure Service Fabric alone or via the Docker Compose support in Azure Service Fabric.

Have cross posted to the Service Fabric team's github to see if they would consider supporting... Add support for autopilot pattern in Docker containers ☺️

[jwreagor]

Interesting that Azure will run on-prem, I feel like that's a big win for security and data ownership.

For anyone following along, it would be beneficial to draw comparisons with other platforms in how they handle service registration/discovery and health checks and expose those APIs. After the holiday I'm planning to push deeper into integration with Consul for blocking queries (#461), watching keys (#506), and multiple watches (#518). I'll keep the interface as open as possible.

Appreciate the suggestion @MedAnd!

[MedAnd]

Hi @cheapRoc , found an Unofficial Service Fabric Golang API which could facilitate the above:

Unofficial Service Fabric Golang API
Træfik extra: Service Fabric Provider

[Smithx10]

@MedAnd

Can someone run Consul in Azure?
What does Azure use for Service Discovery?
Does Azure Provide IP Per Container?

Thank You,
Bruce Smith

[jwreagor]

re: @Smithx10

Can someone run Consul in Azure?

Of course, you can run Consul anywhere.

What does Azure use for Service Discovery?

That's what we're discussing here and possible third party work to use ContainerPilot against another service discovery/registration. At least that's what I was proposing.

Does Azure Provide IP Per Container?

Good question, I would assume so since they also have native container's "as-a service". But the question has no relevance to this discussion.

I think we can both agree to promote libraries across cloud providers, but I currently don't have a utility for it myself. I'm also interested in architectures involving ContainerPilot and AWS Fargate which does have IP-per container AFAIK.

[Smithx10]

@cheapRoc

Does Azure Provide IP Per Container?:
Good question, I would assume so since they also have native container's "as-a service". But the question has no relevance to this discussion.

It might, depending if he wants to use CP in the APP (autopilot pattern). One of the prerequisites is as follows:

Network virtualization that gives each container its own VNIC, eliminates port conflicts, and simplifies communication between containers across different compute nodes."

Obviously I'm assuming here... and that's bad,
/Walks away slowly. :)

[misterbisson]

@MedAnd: I'm still hoping to learn more about Azure Service Fabric and what it offers and how it compares to other solutions. Importantly, what does ContainerPilot add to Azure Service Fabric, and how do discovery solutions in Azure Service Fabric compare to Consul?

This has some similarities with running Autopilot Pattern apps run on Kubernetes, which works well when users run a Consul cluster on Kubernetes. Some people have asked why we can't support Etcd in that context (ContainerPilot used to support Etcd, but that support was removed). The short answer is that Etcd is really there for Kubernetes' exclusive use.

One of the big reasons that we gave up on Etcd support in ContainerPilot was that we increasingly needed to use features only available in Consul. For example, Autopilot Pattern MySQL uses Consul locks to help coordinate the primary and replicas. Etcd can't (and, we understand, won't) support those features. There are other Consul features we wish to use with ContainerPilot that aren't commonly available.

@cheapRoc asked some good questions earlier, and I'd like to dig into those:

it would be beneficial to draw comparisons with other platforms in how they handle service registration/discovery and health checks and expose those APIs

Those questions aren't an objection to integration, but we need to better understand them, because integration is not necessarily a small task:

A quick look at their documentation leads me to believe that much of ContainerPilot's underlying health and service discovery would need to be reworked to accommodate their APIs.

As I asked at the top, it would be very useful to know what ContainerPilot adds to Azure Service Fabric, and how the discovery solutions in Azure Service Fabric compare to Consul.

[MedAnd]

Hi @misterbisson,

Appreciate the opportunity to discuss Azure Service Fabric & my scenario in relation to Containerpilot. Azure Service Fabric being a distributed systems platform & framework for packaging, deploying, scaling and managing stateful and stateless microservices and Docker containers.

I believe Containerpilot support for Azure Service Fabric would facilitate complex clusters of Docker Linux containers and mixed workloads. For example say I have an Azure Service Fabric cluster of 5-10 nodes. I would host my microservices written in .Net Core on this cluster in addition to running a HA Linux Docker PostgreSQL cluster, and an Elasticsearch Linux Docker cluster. Azure Service Fabric & Containerpilot integration would manage node failover of the Docker Linux containers.

Whilst it’s possible to run Docker containers today (Docker Compose v3 is supported) on Azure Service Fabric, there is no good way to control from within the container the actual cluster formation process nor re-use open source HA solutions such: https://github.com/autopilotpattern

Instead of having one orchestrator and one cluster my projects for example need an Azure Service Fabric cluster, a separate Elasticsearch cluster + maybe a Kubernetes PostgreSQL HA cluster… my aim is to consolidate and reuse industry proven solutions which allow my customers agility. For example some of my projects may start running on-prem Azure Service Fabric, but then at some future date the project is lifted and shifted to a cloud provider of choice.

In regards to the technical portion of your question, the Azure Service Fabric SDK unfortunately at this stage does not seem to allow 3rd parties such as yourself to easily integrate. I have created this github issue microsoft/service-fabric-issues#666 and I can see @mani-ramaswamy, @vturecek and @raja-krishnan from the MS Azure Service Fabric team have been assigned. I hope they might be able to answer in more detail and I assume at a minimum it would be good for the MS Azure Service Fabric team to understand what integration points would need to be exposed to support open source solutions such as yours...

PS. @jjcollinge would also know more about the technical integration points and using Golang as he is working on running Træfik (reverse proxy and load balancer) in Azure Service Fabric.

PSS. Unofficial Service Fabric Golang API

[Jaff]

I'm going to chime-in here with a similar request for kubernetes. Trying to bring up containerpilot on a k8s pod, I get a message: time="2022-08-02T18:32:18Z" level=fatal msg="error listening to socket at /var/run/containerpilot.socket: listen unix /var/run/containerpilot.socket: bind: read-only file system"
How can I get past this?

[MartijnDekkers]

I'm going to chime-in here with a similar request for kubernetes. Trying to bring up containerpilot on a k8s pod, I get a message: time="2022-08-02T18:32:18Z" level=fatal msg="error listening to socket at /var/run/containerpilot.socket: listen unix /var/run/containerpilot.socket: bind: read-only file system" How can I get past this?

Perhaps https://stackoverflow.com/questions/49614034/kubernetes-deployment-read-only-filesystem-error applies to your issue, @Jaff