[LinuxOnly] is egregiously misused at least in sig-network tests

[danwinship]

In test/e2e/network, there are currently 45 tests marked [LinuxOnly], of which it seems that 37 are incorrect or at least dubious:

• 15 claim that Windows does not support SessionAffinity, which was implemented in pkg/proxy/winkernel 4 years ago
• 14 test dual-stack, which works on Windows but is (was?) not fully supported by our CI environment
• 3 need UDP support in agnhost, which the tests in question claim does not work, though I suspect this actually means "at the time these tests were merged, the windows agnhost image had not been rebuilt with the recently-added udp client yet".
• 2 use hostNetwork pods, which now exist in alpha on Windows, so we ought to at least optionally be running them there
• 1 uses pod hostPorts, which have been supported on Windows since 2020.
• 1 is skipped because "IPv6 is not supported on Windows", which is not true

The seemingly-correct ones:

• 5 use SCTP, which AIUI the Windows kernel does not implement and does not plan to implement.
• 3 test partial DNS names (kubernetes.default.svc) which apparently are not supported by the Windows resolver

cc @claudiubelu @knabben @MikeZappa87
/sig windows
/sig network

[knabben]

/triage accepted

[claudiubelu]

Hello,

Indeed, there are quite a few tests which are marked as [LinuxOnly], and since then there were some tests which were fixed and / or had this label removed over time, not necessarely all networking-related [1][2][3][4][5][6] (may be others as well). But out of those 45 tests you mentioned, not all of them are marked as [Conformance], or at least as far as I can tell, I see only 8:

ubuntu@ubuntu:~/workdir/kubernetes$ yq '.[] | .codename | select(contains("LinuxOnly")) | select(contains("network"))' test/conformance/testdata/conformance.yaml
[sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
[sig-network] HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol [LinuxOnly] [Conformance]
[sig-network] Networking Granular Checks: Pods should function for node-pod communication: http [LinuxOnly] [NodeConformance] [Conformance]
[sig-network] Networking Granular Checks: Pods should function for node-pod communication: udp [LinuxOnly] [NodeConformance] [Conformance]
[sig-network] Services should be able to switch session affinity for NodePort service [LinuxOnly] [Conformance]
[sig-network] Services should be able to switch session affinity for service with type clusterIP [LinuxOnly] [Conformance]
[sig-network] Services should have session affinity work for NodePort service [LinuxOnly] [Conformance]
[sig-network] Services should have session affinity work for service with type clusterIP [LinuxOnly] [Conformance]

Also worth mentioning that there may be more tests which are being skipped on Windows through something like SkipIfNodeOSDistroIs("windows"), I'm not sure if they're included in your count or not. I remember that we were not allowed to add skips to [Conformance] tests, which is why that label was necessary in the first place. This label was not typically added to non-conformance tests, but IMO, it's more useful to have it as a label; it makes it easier to grep which tests are supposed to be Linux-only or not. As for the number of Linux-only networking tests, a more complete list of tests could be useful, so we can check whether or not they're mislabeled or not, or which tests exactly we're talking about.

Also note that non-Conformance tests are not typically tested as the Conformance ones, so I can't say much about them passing or not. There were some efforts in the past for promoting more networking tests to Conformance [7], but that effort seems discontinued at this point.

Now, regarding the tests you've mentioned:

• SessionAffinity: Probably requires more in-depth investigation. But from a quick look, it doesn't work [8] (marked all networking tests as [Conformance] so they'd be run by the Windows CI):

Kubernetes e2e suite: [It] [sig-network] Services should have session affinity work for NodePort service [Conformance] 2m47s
{ failed [FAILED] Affinity should hold but didn't.
In [It] at: k8s.io/kubernetes/test/e2e/network/service.go:266 @ 04/22/24 19:31:21.999
}

Kubernetes e2e suite: [It] [sig-network] Networking Granular Checks: Services should function for client IP based session affinity: udp [Conformance]   1m6s
{ failed [FAILED] Unexpected endpoints return: map[netserver-0:{} netserver-1:{}], expect 1 endpoints
In [It] at: k8s.io/kubernetes/test/e2e/network/networking.go:447 @ 04/22/24 19:34:02.49
}

• dual-stack: Indeed, according to James [9], these tests could be enabled for WS 2022 jobs but not for WS 2019 jobs. I assume there should be a dedicated dual-stack WS 2022 job for this, which would also require e2e to receive a dual-stack-related config flag. From what I can see, they're labeled as [Feature:IPv6DualStack], so we can use that label to exclude it from other jobs.

• UDP support in agnhost: not sure which tests are those. should function for node-pod communication: udp?

• hostNetwork tests: agreed. Though, I thought there were more than 2 tests.

• hostPorts test seems to be failing on Windows [7], may require further investigation:

Kubernetes e2e suite: [It] [sig-network] HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol [Conformance]   45s
{ failed [FAILED] Failed to connect to exposed host ports
In [It] at: k8s.io/kubernetes/test/e2e/network/hostport.go:161 @ 04/22/24 19:29:22.484
}

• IPv6 test: I assume you refer to should provide Internet connection for containers? It might require a bit more investigation. It seems that there's a failure [10]: forward host lookup failed: h_errno 11001: HOST_NOT_FOUND

• SCTP: indeed, it seems that Windows doesn't have it [11]. But there seem to be some 3rd party implementations. WDYT, should we consider one of those and say it's officially supported on Kubernetes for Windows?

• partial DNS names: yes, that is correct. You may either use only the FQDNs, or only the hostname part.

[1] #101063
[2] #72729
[3] #97045
[4] #85453
[5] #78731
[6] #75591

[7] #73425
[8] https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/124447/pull-kubernetes-e2e-capz-windows-master/1782468233807269888
[9] #100870 (comment)
[10] https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/124447/pull-kubernetes-e2e-capz-windows-master/1782725964745150464
[11] https://learn.microsoft.com/en-us/answers/questions/778329/sctp-driver

[danwinship]

But out of those 45 tests you mentioned, not all of them are marked as [Conformance]

Sure, but they all involve documented features that users might want to use. And as you said, we do eventually want to move more tests toward Conformance, so Windows really ought to be aiming to pass everything that isn't truly linux-specific.

Also worth mentioning that there may be more tests which are being skipped on Windows through something like SkipIfNodeOSDistroIs("windows"), I'm not sure if they're included in your count or not.

No... I was only looking at [LinuxOnly].

• UDP support in agnhost: not sure which tests are those. should function for node-pod communication: udp?

No, sorry, I should have listed these out. The NetworkPolicy between server and client using UDP tests in test/e2e/network/netpol/network_policy.go. (In fact, they are both [LinuxOnly] and SkipIfNodeOSDistroIs("windows")!)

• hostNetwork tests: agreed. Though, I thought there were more than 2 tests.

Perhaps the others are skipped rather than labeled.

• IPv6 test: I assume you refer to should provide Internet connection for containers?

Yes

• SCTP: indeed, it seems that Windows doesn't have it [11]. But there seem to be some 3rd party implementations. WDYT, should we consider one of those and say it's officially supported on Kubernetes for Windows?

SCTP isn't even implemented by most Linux network plugins and it seems like sig-windows already has too much to do anyway, so it seems reasonable to just say that it's not supported on Windows. If someone wants to get it to be officially-supported then they can do the work...

[sebsoto]

/assign @sebsoto