Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eksctl delete cluster should not drain nodes #7719

Open
cPu1 opened this issue Apr 22, 2024 · 5 comments
Open

eksctl delete cluster should not drain nodes #7719

cPu1 opened this issue Apr 22, 2024 · 5 comments

Comments

@cPu1
Copy link
Collaborator

cPu1 commented Apr 22, 2024

When deleting a cluster, eksctl drains all nodes before proceeding with nodegroup deletion. This behaviour is of little use and further increases the duration of the command as the workloads will remain unschedulable after pod termination.

Starting with CoreDNS v1.9.3-eksbuild.5 and v1.10.1-eksbuild.2, an addon that is installed by default, a PodDisruptionBudget with maxUnavailable: 1 is also created, preventing the nodes from being drained as attempting to evict more than one CoreDNS pod will violate the PodDisruptionBudget. While this can be worked around by passing --disable-nodegroup-eviction, eksctl delete cluster should work without needing any additional arguments in most cases.

We may optionally want to still gracefully terminate pods to give pods a chance to clean up any external resources.

Related: #6287

@rglonek
Copy link

rglonek commented Apr 23, 2024

+1 we have short-lived EKS clusters used for testing k8s stuff. We create and destroy them. At least give an option to in eksctl cluster destroy to skip draining, we don't care about that, just want the cluster gone. eksctl cluster delete -n NAME --no-drain maybe?

@cPu1
Copy link
Collaborator Author

cPu1 commented Apr 23, 2024

+1 we have short-lived EKS clusters used for testing k8s stuff. We create and destroy them. At least give an option to in eksctl cluster destroy to skip draining, we don't care about that, just want the cluster gone. eksctl cluster delete -n NAME --no-drain maybe?

@rglonek there is already an option for skipping pod eviction by passing --disable-nodegroup-eviction to eksctl delete cluster. This issue for having that as the default behaviour.

@rglonek
Copy link

rglonek commented Apr 24, 2024

It skips node eviction, but still runs through the draining process, no? The logs seem to indicate so, unlike when I do eksctl delete nodegroup ... --drain=false. Would be nice if we had --drain-nodegroup=false option in eksctl delete cluster :)

@tanvp112
Copy link

tanvp112 commented Apr 25, 2024

This is not a bug. Changing this default behavior is a bad idea because "incident" do happens! Explicit use of --disable-nodegroup-eviction --force is a fully conscious approach and will delete the cluster successfully despite the disruption budget.

Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions github-actions bot added the stale label May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants