MTU issue for large UDP packets with native routing and DSR #32601
Labels
area/loadbalancing
Impacts load-balancing and Kubernetes service implementations
feature/dsr
Relates to Cilium's Direct-Server-Return feature for KPR.
info-completed
The GH issue has received a reply from the author
kind/community-report
This was reported by a user in the Cilium community, eg via Slack.
kind/question
Frequently asked questions & answers. This issue will be linked from the documentation's FAQ.
needs/triage
This issue requires triaging to establish severity and next steps.
sig/datapath
Impacts bpf/ or low-level forwarding details, including map management and monitor messages.
Is there an existing issue for this?
What happened?
We have an environment with DSR and native routing, and a UDP service (SIP) with 3 endpoints, running on 3 different nodes. The MTU on the network is 1500 bytes. When a large, external 1500 byte UDP packet gets sent to the service, it gets routed to one of the nodes, which DNATs it to one of the pods. If the pod is running on a different node, it gets sent over the network to another node, and because of DSR, an 8 byte header gets added to the packet, making it 1508 bytes long, which is higher than the MTU. And in the end, some entity (either the host, or the switch [we have an L2 network], I didn't really look into that) drops the packet.
For TCP, this is not an issue as far as I know, since this 8 byte header only gets added to packets with the SYN flag, which are almost never large.
Cilium Version
Client: 1.15.1 a368c8f 2024-02-14T22:16:57+00:00 go version go1.21.6 linux/amd64
Daemon: 1.15.1 a368c8f 2024-02-14T22:16:57+00:00 go version go1.21.6 linux/amd64
Kernel Version
5.14.0-362.24.1.el9_3.0.1.x86_64
Kubernetes Version
v1.26.15+rke2r1
Regression
No response
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Cilium Users Document
Code of Conduct
The text was updated successfully, but these errors were encountered: