Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cilium-cni: Reserve ports that can conflict with transparent DNS proxy #32128

Merged

Conversation

gandro
Copy link
Member

@gandro gandro commented Apr 22, 2024

This commit adds the ability for the cilium-cni plugin to reserve IP ports (via the ip_local_reserved_ports sysctl knob) during CNI ADD. Reserving these ports will prevent the container network namespace to use these ports as an ephemeral source port, while still allowing the port to be explicitly allocated (see [1]).

This functionality is added as a workaround for #31535 where transparent DNS proxy mode causes conflicts when an ephemeral source port is being chosen that is already being used in the host network namespace. By reserving ports used by Cilium itself (such as WireGuard and VXLAN), we hopefully reduce the number of such conflicts.

The set of reserved ports can be configured via a newly introduced agent flag. By default, it will reserve an auto-generated set of ports. The list of ports is configurable such that users running custom UDP services on ports in the ephemeral port range can provide their own set of ports. The flag may be set to an empty string to disable reservations altogether.

[1] https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt

@gandro gandro added area/cni Impacts the Container Networking Interface between Cilium and the orchestrator. release-note/bug This PR fixes an issue in a previous release of Cilium. backport/author The backport will be carried out by the author of the PR. needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch area/fqdn Affects the FQDN policies feature needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch feature/wireguard Relates to Cilium's Wireguard feature needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Apr 22, 2024
@gandro gandro force-pushed the pr/gandro/transparent-dns-proxy-reserve-known-ports branch from 7a814da to c7afe2f Compare April 22, 2024 15:27
@gandro gandro marked this pull request as ready for review April 22, 2024 15:35
@gandro gandro requested review from a team as code owners April 22, 2024 15:35
@gandro
Copy link
Member Author

gandro commented Apr 22, 2024

I did some manual testing by forcing WireGuard conflicts (port 51871) by reducing the ephemeral port range via echo 51871 51874 > /proc/sys/net/ipv4/ip_local_port_range on the client pod network namespace. With ip_local_reserved_ports set to 8472,51871, there are no errors. As soon as I unset ip_local_reserved_ports (in the pod namespace), dig requests start failing.

@gandro
Copy link
Member Author

gandro commented Apr 22, 2024

/test

plugins/cilium-cni/cmd/cmd.go Show resolved Hide resolved
pkg/option/config.go Show resolved Hide resolved
@gandro gandro force-pushed the pr/gandro/transparent-dns-proxy-reserve-known-ports branch from c7afe2f to d819ae1 Compare April 23, 2024 09:08
@gandro
Copy link
Member Author

gandro commented Apr 23, 2024

Thanks for the feedback! I've replied inline, hope that answers some of the questions.

@gandro
Copy link
Member Author

gandro commented Apr 23, 2024

/test

Copy link
Contributor

@learnitall learnitall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API changes look good, thank you!

@gandro gandro removed the needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch label May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.15 in 1.15.5 May 8, 2024
@gandro gandro added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. and removed needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch labels May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.14 in 1.14.11 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.13 in 1.13.16 May 8, 2024
gandro added a commit to gandro/cilium that referenced this pull request May 8, 2024
This is a follow-up suggested by Julian in a previous PR:
cilium#32128 (comment)

Fixes: 11fe7cc ("cilium-cni: Reserve ports that can conflict with transparent DNS proxy")

Note: This fix has already been applied to the backported version of the
above commit.

Suggested-by: Julian Wiedmann <jwi@isovalent.com>
Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
@github-actions github-actions bot added backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. and removed backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. labels May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.14 in 1.14.11 May 8, 2024
@github-actions github-actions bot added backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. and removed backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.14 in 1.14.11 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.13 to Backport done to v1.13 in 1.13.16 May 8, 2024
github-merge-queue bot pushed a commit that referenced this pull request May 14, 2024
This is a follow-up suggested by Julian in a previous PR:
#32128 (comment)

Fixes: 11fe7cc ("cilium-cni: Reserve ports that can conflict with transparent DNS proxy")

Note: This fix has already been applied to the backported version of the
above commit.

Suggested-by: Julian Wiedmann <jwi@isovalent.com>
Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
jshr-w pushed a commit to jshr-w/cilium that referenced this pull request May 16, 2024
This is a follow-up suggested by Julian in a previous PR:
cilium#32128 (comment)

Fixes: 11fe7cc ("cilium-cni: Reserve ports that can conflict with transparent DNS proxy")

Note: This fix has already been applied to the backported version of the
above commit.

Suggested-by: Julian Wiedmann <jwi@isovalent.com>
Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
@@ -604,13 +628,12 @@ func (cmd *Cmd) Add(args *skel.CmdArgs) (err error) {

if err = ns.Do(func() error {
if ipv6IsEnabled(ipam) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like local IP ports are reserved only if IPv6 is enabled?

@julianwiedmann julianwiedmann added the area/proxy Impacts proxy components, including DNS, Kafka, Envoy and/or XDS servers. label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cni Impacts the Container Networking Interface between Cilium and the orchestrator. area/fqdn Affects the FQDN policies feature area/proxy Impacts proxy components, including DNS, Kafka, Envoy and/or XDS servers. backport/author The backport will be carried out by the author of the PR. backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. feature/wireguard Relates to Cilium's Wireguard feature ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/bug This PR fixes an issue in a previous release of Cilium.
Projects
No open projects
1.13.16
Backport done to v1.13
1.14.11
Backport done to v1.14
1.15.5
Backport done to v1.15
Development

Successfully merging this pull request may close these issues.

None yet

8 participants