Kubernetes's stock scheduler won't take into account the current utilization of nodes in cluster while making scheduling decisions. We intend to solve this problem with kube-usage-scheduler-extender. It is designed to be an add-on service on top of kubernetes to enhance scheduler with resource management capabilities. It has two components:
- Kube metrics client to get node level metrics in a kubernetes cluster. As of now, the code returns the node with least CPU utilization but this could be extended to any resource(like memory, GPU etc). We are relying on metrics-server to get information related to node usage(https://github.com/kubernetes-incubator/metrics-server).
- A sample kubernetes scheduler extender which returns the node with least cpu utilization in the cluster.
- Kubernetes scheduler has the concept of scheduler extender where it sends the request to HTTP server before binding a pod to node if extender is enabled. The request sent to HTTP server includes pod and nodelist that are filtered.
- The extender as of now has a filter function which further filters the nodes from nodelist supplied. The filtering is based on computation algorithm. The computation algorithm as of now is a very simple algorithm which talks to metrics server and get the node information for CPU, memory usage.
- Once the filtering happens, the HTTP server responds back with node which has the least CPU utilization.
- Make sure that metrics-server is running as deployment and getting node level metrics. You can test this using:
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" | jq. This should return information related to all the nodes along with current usage on the nodes. - Do a git clone of this repo and then run:
$ make
and then run kube-usage-scheduler-extender:
$ _output/bin/kube-ext --kubeconfig <path to kubeconfig file>
This is not ready for production use yet. But give it a spin and provide some feedback.