New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support LoadBalancer Services #4
Comments
Preliminary ResearchI started examining this problem shortly after releasing Kubernaut. Unfortunately, I have discovered there is no simple ideal solution to this problem. The Primary ProblemKubernaut does not support
Problem 2This is a small problem and is internal to our operation of the service. Amazon ELB instances are NOT free. They are charged at $0.025/hr (~$18.30/mo) just to run. Data is charged $0.008/GB IN/OUT data. At a minimum, assuming 1 user is consistently using the service for a whole month we have a spend of $18/user just for the ELB. Data is cheap, and they would need to pump a lot of data through the system to cost us a lot of money, but lets once again assume humans are dumb or malicious and someone decides to upload a 100MB archive in a loop through their service. We need to have monitoring in place that allows us to basically cut off the user. That is engineering effort to us to prevent someone from causing financial harm. Problem 3I examined whether it was possible to disable the ELB provisioning functionality in the AWS cloud provider integration for Kubernetes and the answer is that it is not possible. My idea was to disable it and replace it with a different service controller that would talk to say a HAProxy or Nginx cluster we ran that could act as a multi-tenant TCP load balancer. A brief technical explanation:
If we want this kind of customizability it seems we need to get involved in Kubernetes development process:
|
Doing additional implementation research. To avoid hard AWS limits we will need to configure the following:
|
We will also need to adjust our usage of https://kubernetes.io/docs/admin/kubeadm/ Specifically see "Cloudprovider integrations (experimental)" |
Suggested approach I learned of at Velocity NYC from Kelsey Hightower
|
Discovered this while poking around for AWS and Kubernetes stability issues... this would be an additional implementation problem: kubernetes/kubernetes#29298 |
Kubernaut currently does not support LoadBalancer services but many people use
type: LoadBalancer
in their service manifests. The current work around is to usetype: NodePort
but this means changing manifests specifically for Kubernaut which is undesirable.The text was updated successfully, but these errors were encountered: