kubelet service doesn't start on a system with swap

[MasseGuillaume]

Steps to reproduce

  # machine with swapDevices

  # kubernetes default config
  services = {
    kubernetes = {
      roles = ["master" "node"];
    };
  };

 sudo journalctl -u kubelet.service | grep "failed to run"

Nov 15 20:10:05 nixos kubelet[1391]: F1115 20:10:05.994998    1391 server.go:262] failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename                                Type                Size        Used        Priority /dev/sda2                               partition        8388604        0        -2]

More details: kubernetes/kubeadm#610

Workaround: Disable swap: sudo swapoff -a & Restart service: sudo systemctl restart kubelet.service

[c0bw3b]

/cc @johanot @offlinehacker

[johanot]

@MasseGuillaume You can set services.kubernetes.kubelet.extraOpts = ["--fail-swap-on=false"]; to disable this kubelet behavior.

See: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

[MasseGuillaume]

@johanot

you mean: services.kubernetes.kubelet.extraOpts = "--fail-swap-on=false";

(it's a string for some reason)

nixpkgs/nixos/modules/services/cluster/kubernetes/default.nix

Lines 479 to 483 in 21517c6

	extraOpts = mkOption {
	description = "Kubernetes scheduler extra command line options.";
	default = "";
	type = types.str;
	};

I think ideally this should be in the default configuration if we detect a swap device. This way, it would work out of the box.

[johanot]

It's a string yes. Sorry.

However I don't agree that we should change the default behavior of components, unless there's a strong reason for doing so. If the Kubernetes maintainers recommend against running the kubelet on swap-enabled nodes, it's not up to us to nullify this.

[srhb]

Agreed, the point of the upstream setting is presumably to protect against users accidentally runny kubelets on swap-enabled machines by accident. Inverting the default would void this safety. Let's not do that unless we have very good reason. :)