Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support inject multiple cluster-dns-ips to kubelet-config.json #4934

Open
arestuzi opened this issue Oct 26, 2023 · 4 comments
Open

Support inject multiple cluster-dns-ips to kubelet-config.json #4934

arestuzi opened this issue Oct 26, 2023 · 4 comments
Labels
feature New feature or request good-first-issue Good for newcomers

Comments

@arestuzi
Copy link

Description

What problem are you trying to solve?

  1. I want to inject two dns server to pod for backup purpose. One is the coredns service ip address and the other is the localdns service ip address. If the coredns timed out, we can use secondary dns server for resolution.

  2. I tried to configure multiple clusterDNS via karpenter provision. Only the first cluster dns injected to kubelet-config.json after scale up by the karpenter.

cat provision.yaml 
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]
  limits:
    resources:
      cpu: 1000
  providerRef:
    name: default
  consolidation: 
    enabled: true
  kubeletConfiguration: 
    clusterDNS: ["10.0.1.100", "10.0.1.101"]
  1. Via the source code of karpenter and bootstrap. we can see the karpenter selected the first entry and pass it as an argument to bootstrap.
  2. eksctl support multiple cluster dns configuration and it works
    https://eksctl.io/usage/nodegroup-customize-dns/

How important is this feature to you?

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
@arestuzi arestuzi added the feature New feature or request label Oct 26, 2023
@njtran njtran added the needs-investigation Issues that need to be investigated before triaging label Oct 31, 2023
@ipyker
Copy link

ipyker commented Apr 15, 2024

This is important!

@jonathan-innis
Copy link
Contributor

jonathan-innis commented Apr 18, 2024

@ipyker Looks like this is something that would be possible in AL2023 using our kubelet stanza today. Doesn't look like it's possible with AL2 because of limitations around the bootstrap script. I think we should also look into this being possible on BR -- since it looks like we could just convert the BR setting from a string to a slice of strings and everything should just work (assuming that we inject single values into a slice as well). We'd happily accept a PR to add this functionality if this was something that you would be interested in working on.

@jonathan-innis jonathan-innis removed the needs-investigation Issues that need to be investigated before triaging label Apr 18, 2024
@jonathan-innis
Copy link
Contributor

Actually, looking over the bootstrap script and the latest PRs, looks like this is now possible with this change: https://github.com/awslabs/amazon-eks-ami/pull/1685/files

@jonathan-innis jonathan-innis added the good-first-issue Good for newcomers label Apr 18, 2024
@xahhy
Copy link

xahhy commented Apr 29, 2024

@jonathan-innis Hi, we are trying to support multiple DNS as well. And confirmed the eks ami have already support it. I have tried to config it as clusterDNS: ["10.0.1.100,10.0.1.101"] but due to the Karpenter code here


which treated this IP as a IPv6 IP and added --ip-family ipv6 in the bootstrap script and cause node can't join our EKS cluster. So we need either Karpenter to check the ipv6 more smarter or directly support multiple clusterDNS IPs

Created a PR here #6117 for quick fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request good-first-issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants