[ERROR Swap]: running with swap on is not supported. Please disable swap

[cjdcordeiro]

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

kubeadm version (use kubeadm version): 1.9.0

Environment:

• Kubernetes version (use kubectl version): 1.9.0
• Cloud provider or hardware configuration: Raspberry Pi
• OS (e.g. from /etc/os-release): Raspbian GNU/Linux 8 (jessie)
• Kernel (e.g. uname -a): Linux 4.9.35-v7+
• Others:

What happened?

When doing kubeadm init the error pops up [ERROR Swap]: running with swap on is not supported. Please disable swap. The suggested fix is to use kubelet's flag so basically:

kubeadm reset 
echo 'Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"' >> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

systemctl daemon-reload
systemctl restart kubelet

kubeadm init

But in the end, the error is still there. Maybe this is a Kubelet issue instead or maybe I'm misconfiguring something...

[luxas]

If you do set the kubelet flag, you're safe to do this kubeadm init --ignore-preflight-errors Swap, but I'd recommend you to just turn off swap instead, as you're going into unsupported territory here.

[cjdcordeiro]

no need to re-open this but I still feel this is a bit counter intuitive. If you set the kubelet flag, then I would expect I don't need to ignore the preflight errors. If --ignore-preflight-errors is not passed, the error message remains the same whether the kubelet flag is enabled or not, and that's misleading.

[cli0]

What is the command to turn off swap?

[cjdcordeiro]

swapoff -a

[kachkaev]

It's true that swapoff -a is a silver bullet in most cases, however, certain k8s setups may really require swap. For instance, I've got a very small and cheap VM with just 1GB RAM, which I use for a personal GitLab Runner that rarely handles short CI/CD tasks. If I increase the size of the machine, I'll be paying more for a resource that's 99% idle. If I disable swap, npm install and other scripts inside the buid pods may hang because they require quite a lot of memory, although for short periods of time. Thus, a single-node kubeadm cluster with gitlab runner chart and swap is what suits me best.

Here is how I could get my mini-cluster up and running:

UPD: Below workaround applies to k8s 1.10- only - for 1.11+ see https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ (TLDR: you can specify kubeletConfiguration:\n failSwapOn: false in your kubeadm's config.yaml and then kubeadm init --config config.yaml --ignore-preflight-errors Swap).

kubeadm reset 

## ↓ see explanation below
sed -i '9s/^/Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"\n/' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

systemctl daemon-reload
systemctl restart kubelet

echo "
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
api:
  bindPort: ${K8S_API_PORT}
apiServerCertSANs: ${K8S_API_EXTRA_HOSTS}
" > /tmp/config.yaml

kubeadm init --config /tmp/config.yaml --ignore-preflight-errors Swap

## make possible to run workload on master
kubectl taint nodes --all node-role.kubernetes.io/master-

Below paragraphs apply to k8s 1.10- only
The reason why I used sed -i '9s/^/... instead of echo 'Environment="..."' >> ... as mentioned by @cjdcordeiro is because in the latter case the lines in 10-kubeadm.conf stacked in the wrong order:

...
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ$
Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"

Because KUBELET_EXTRA_ARGS appeared after ExecStart, it looked like it was not picking up. With sed -i '9s/^/..., file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf ends up like so and works:

...
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
Environment="KUBELET_EXTRA_ARGS=--fail-swap-on=false"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ$

---

It'd be great if enabling swap in kubeadm was easier than now – this would save people across the world tones of hours. Making my mini-cluster work after upgrading to 1.8 was a real pain because I'm quite inexperienced in Linux administration and I think it'd be great if others did not have to take the same path. The ideal solution would look like this IMO:

echo "
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
kubeletConfiguration:
  allowSwap: true
" > /tmp/config.yaml

kubeadm init --config /tmp/config.yaml

Of course, enabling swap should remain an edge case because it may bite in lots of circumstances. However, it'd be great if kubeadm users had a choice. Until then, it'd be great if there was an opened issue about enabling swap.

[mcronce]

+1 for having an option for an experimental/alpha/etc "feature" flag to enable swap that adds it to the kubeadm drop-in in kubelet.service.d and disables that pre-flight check. Definitely an edge case, agreed, but it would be nice if it were easier.

(Googling and being able to quickly find this issue helps a lot, though.)

[geerlingguy]

See related: kubernetes/kubernetes#53533

[yeison]

I turned my swap off but I'm still getting this error.

[rcmorano]

Just for the record, at least in an ubuntu-based system, I guess the place to add the --fail-swap-on=false flag is in /etc/default/kubelet file; not in the systemd conf file itself.

[neolit123]

Just for the record, at least in an ubuntu-based system, I guess the place to add the --fail-swap-on=false flag is in /etc/default/kubelet file; not in the systemd conf file itself.

for 1.11+ this is true.

[bor8]

To disable it permanently, just insert @reboot /sbin/swapoff -a with a line break at the end in sudo crontab -e.

Tested on Ubuntu 16.04 and 18.04.