Discussion roadmap v1.0

[pablodav]

I'm creating milestone v1.0. https://github.com/pablodav/kubernetes-for-windows/milestone/1

I have finally got merged into kubespray the required support for this project:

kubernetes-sigs/kubespray#3200 (comment)

These are very good news now, as we can start focusing more on windows requirements.

Also implemented upgrade to have support for kubernetes v1.11.2 (with kubespray + updates in this project).

Let's have this thread for discussion.

ping @ptylenda

[pablodav]

related to: #8

[pablodav]

ping @damoxc

[damoxc]

I still need to familiarise myself with both ansible and kubespray, but in my eyes a 1.0 would be a version that can reliably build-up a cluster containing Windows nodes in a fixed, but functional, configuration. Flexibility falling in the post 1.0 era.

[ptylenda]

Looks like we have some support official support for CNI on windows finally, let's hope it will work without any workarounds: containernetworking/plugins#193

BTW the biggest issue to me with kubernetes for windows is inherent problems with HNS networking and NAT vs internal cluster communication problems. I expect that after last 4-5 months, fixes have been released to the latest version of insider builds of Windows and together with new CNI we can get a good setup for migrating existing windows apps/services to k8s. We should also fully support vxlan and host-gw plugins, as far as I remember, I have managed to get host-gw working only, whereas vxlan had some bugs in CNI plugins. I expect that this is also fixed with the above PR.

Apart from that for 1.0 milestone, it may be a good idea to adjust windows services management, in k8s 1.10 there was a pull request for native services support (without nssm + ansible of course) for kubelet: kubernetes/kubernetes#60144

[pablodav]

Great, good news.
I agree with the native services support.

[daschott]

Awesome, good to see this! 🎉

@ptylenda both internal cluster communication via pods or services, as well as outbound NAT works today simultaneously. But if you are using VM's you need to enable Mac spoofing for your VM network adapter, else cluster communication will not work.

If outbound communication isn't working I would check the ExceptionList l2bridge.conf or cni.conf which specifies an OutboundNAT Exception list where we don't want NAT'ing to occur. Usually something is wrong there when people report outbound connectivity issues.

Also, please do not use ping <IP> for anything that requires NAT to demonstrate connectivity because there are no ICMP rules programmed in VFP (Windows equivalent of iptables). Please use TCP/UDP protocol, or curl <IP> equivalent.

[daschott]

There is a basic band-aid guide here to deploy using flannel (host-gw) mode, but I'm hoping to update official docs as soon as I can.

[pablodav]

Kubernetes SIG-Windows group is doing lot of work for 1.14:

https://github.com/orgs/kubernetes/projects/8

kubernetes/enhancements#116

Looks like it would be in better shape.

[pablodav]

Lot of things have been improved with k8s 1.14 and windows 2019 ltsc.

https://kubernetes.io/docs/setup/windows/#supported-functionality

Looks like it should be the target version to support.