Prepare for release v1.13.4

[michi-covalent]

Summary of Changes

Minor Changes:

• Add agent flag enable-ipsec-key-watcher to allow users to disable the IPsec key watcher and thus require an agent restart for the key rotation to take effect. (Backport PR v1.13 Backports 2023-06-07 #25977, Upstream PR ipsec: Flag to disable key watcher and Helm values #25893, @pchaigno)
• Updating documentation helm values now works also on arm64. (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR docs: Remove non-x86 restriction #25422, @jrajahalme)

Bugfixes:

• Add drop notifications for various error paths in the datapath. (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR bpf: add missing drop notifications #25183, @julianwiedmann)
• bpf,datapath: read jiffies from /proc/schedstat (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR bpf,datapath: read jiffies from /proc/schedstat #25795, @ti-mo)
• Compare annotations before discarding CiliumNode updates. (Backport PR v1.13 Backports 2023-05-22 #25588, Upstream PR Compare annotations before discarding CiliumNode updates. #25465, @LynneD)
• CPU overhead regression introduced in v1.13 is fixed. (daemon: Don't register monitor listener for auth required events #25548, @jrajahalme)
• Fix a bug due to which we would leak Linux XFRM policies, potentially leading to increased CPU consumption, when IPsec is enabled with Azure or ENI IPAM. (Backport PR v1.13 backports 2023-06-05 #25897, Upstream PR ipsec: Fix leak of XFRM OUT policies #25784, @pchaigno)
• Fix a bug that would cause connectivity drops of type XfrmInNoStates on upgrade when IPsec is enabled with ENI or Azure IPAM mode. (Backport PR v1.13 backports 2023-06-05 #25897, Upstream PR ipsec: Fix XfrmInNoStates drops on ESK & AKS upgrades #25724, @pchaigno)
• Fix a bug that would cause connectivity drops of type XfrmOutPolBlock on upgrade when IPsec is enabled. (Backport PR v1.13 backports 2023-06-05 #25897, Upstream PR ipsec: Fix XfrmOutPolBlock drops on upgrades #25735, @pchaigno)
• Fix a possible deadlock when using WireGuard transparent encryption. (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR wireguard, linuxnodehandler: untangle WireGuard agent from the linux node handler #25419, @bimmlerd)
• Fix a possible deadlock when using WireGuard transparent encryption. (Backport PR [v1.13] Author backports of Node Handler for wireguard, and dependent PR #25923, Upstream PR wireguard, linuxnodehandler: untangle WireGuard agent from the linux node handler #25419, @bimmlerd)
• Fix bug affecting EKS installations with IPsec encryption enabled, where Cilium wouldn't attach its IPsec BPF program to new ENI interfaces, resulting in connectivity loss between pods on remote nodes. (Backport PR v1.13 backports 2023-06-05 #25897, Upstream PR ipsec: Reinitialize IPsec for new devices in ENI mode #25744, @joamaki)
• Fix downgrade path from 1.14 to 1.13 due to stale IPAM-allocated IPv6 on cilium_host (Delete stale v6 address from cilium_host #25962, @jschwinger233)
• Fix incorrect hubble flow data when HTTP requests contain an x-forwarded-for header by adding an explicit use_remote_address: true config to Envoy HTTP configuration to always use the actual remote address of the incoming connection rather than the value of x-forwarded-for header, which may originate from an untrusted source. This change has no effect on Cilium policy enforcement where the source security identity is always resolved before HTTP headers are parsed. Previous Cilium behavior of not adding x-forwarded-for headers is retained via an explicit skip_xff_append: true config setting, except for Cilium Ingress where the source IP address is now appended to x-forwarded-for header. (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR envoy: Never use x-forwarded-for header, add for Cilium Ingress #25674, @jrajahalme)
• Fix leak of IPsec XFRM FWD policies in IPAM modes cluster-pool, kubernetes, and crd when nodes are deleted.
  Fix incorrect catch-all default-drop XFRM OUT policy for IPsec IPv6 traffic that could lead to leaking plain-text IPv6 traffic if combined with some other bug. (Backport PR v1.13 backports 2023-06-09 #26079, Upstream PR ipsec: Change XFRM FWD policy to simplest wildcard #25953, @pchaigno)
• Fix missing drop notifications on conntrack lookup failures when IPv4 and IPv6 are both enabled or socket-level load balancing is disabled. (Backport PR v1.13 Backports 2023-05-22 #25588, Upstream PR Issue #22527 - bpf_lxc.c: do drop notifications at drop source #25426, @bleggett)
• Fix RevSNAT for ICMPv6 packets. (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR bpf: nat: fix ICMPv6 ECHO types in snat_v6_rewrite_ingress() #25306, @julianwiedmann)
• Fix three issues in the bug fix to attach IPsec BPF programs to ENI interfaces: do not fatal if loading unexpectedly fails (which may happen if the device is suddenly deleted), ignore veth device changes in order not to reinitialize when new endpoints appear and wait 1 second for further device state changes between reinitializations. (Backport PR v1.13 Backports 2023-06-07 #25977, Upstream PR ipsec: Fix IPsec reinitialization #25936, @joamaki)
• Fixed Cilium agent crash when policy refers to a non-existing Envoy listener. (Backport PR v1.13 backports 2023-06-09 #26079, Upstream PR proxy: Do not panic on local error #25969, @jrajahalme)
• gateway-api: Race condition between routes and Gateway (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR gateway-api: Race condition between routes and Gateway #25573, @sayboras)
• gateway-api: Skip reconciliation for non-matching controller routes (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR gateway-api: Skip reconciliation for non-matching controller routes #25549, @sayboras)
• helm: Correct typo in Ingress validation (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR helm: Correct typo in Ingress validation #25570, @sayboras)
• Reject incorrect configuration enable-host-legacy-routing=false kube-proxy-replacement=partial. (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR daemon: Reject BPF Host Routing without KPR=strict #25803, @pchaigno)

CI Changes:

• [v1.13 backport] test: Switch target FQDN ([v1.13 backport] test: Switch target FQDN #25584, @nbusseneau)
• Add github workflow to push development helm charts to quay.io (Backport PR [v1.13] Backport print-chart-version script for dev Helm chart push #26087, Upstream PR Add github workflow to push development helm charts to quay.io #25205, @chancez)
• hostfw tests flake workaround (Backport PR v1.13 Backports 2023-05-22 #25588, Upstream PR hostfw tests flake workaround #25323, @tommyp1ckles)
• Pick up the latest startup-script image (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR Pick up the latest startup-script image #25774, @michi-covalent)
• test/k8s: add host firewall workaround for svc host policy test. (Backport PR v1.13 Backports 2023-05-22 #25588, Upstream PR test/k8s: add host firewall workaround for svc host policy test. #25461, @tommyp1ckles)
• test/k8s: for services test, wait for all applied manifests to delete (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR test/k8s: for services test, wait for all applied manifests to delete #25341, @tommyp1ckles)
• test/k8s: quarantine K8sDatapathServicesTest (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR test/k8s: quarantine K8sDatapathServicesTest #25670, @aanm)
• test/k8s: update host policies for firewall tests. (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR test/k8s: update host policies for firewall tests. #25374, @tommyp1ckles)
• test: delete ginkgo test "NodePort with L7 Policy from outside" (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR test: delete ginkgo test "NodePort with L7 Policy from outside" #25702, @jschwinger233)
• test: prevent panic on k8s services host fw test on some runs. (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR test: prevent panic on k8s services host fw test on some runs. #25747, @tommyp1ckles)

Misc Changes:

• bpf: dsr: fix typo in tail_nodeport_dsr_ingress_ipv4() (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR bpf: dsr: fix typo in tail_nodeport_dsr_ingress_ipv4() #25742, @julianwiedmann)
• chore(deps): update all github action dependencies (v1.13) (patch) (chore(deps): update all github action dependencies (v1.13) (patch) #25704, @renovate[bot])
• chore(deps): update cilium/actions-app-token action to v0.21.1 (v1.13) (chore(deps): update cilium/actions-app-token action to v0.21.1 (v1.13) #25865, @renovate[bot])
• chore(deps): update dependency cilium/hubble to v0.11.6 (v1.13) (chore(deps): update dependency cilium/hubble to v0.11.6 (v1.13) #26042, @renovate[bot])
• chore(deps): update docker.io/library/alpine docker tag to v3.17.3 (v1.13) (chore(deps): update docker.io/library/alpine docker tag to v3.17.3 (v1.13) #25852, @renovate[bot])
• chore(deps): update docker.io/library/alpine docker tag to v3.17.3 (v1.13) (chore(deps): update docker.io/library/alpine docker tag to v3.17.3 (v1.13) #25853, @renovate[bot])
• chore(deps): update docker.io/library/golang docker tag to v1.19.10 (v1.13) (chore(deps): update docker.io/library/golang docker tag to v1.19.10 (v1.13) #25857, @renovate[bot])
• chore(deps): update docker.io/library/ubuntu:22.04 docker digest to ac58ff7 (v1.13) (chore(deps): update docker.io/library/ubuntu:22.04 docker digest to ac58ff7 (v1.13) #25547, @renovate[bot])
• chore(deps): update quay.io/cilium/hubble docker tag to v0.11.6 (v1.13) (chore(deps): update quay.io/cilium/hubble docker tag to v0.11.6 (v1.13) #25997, @renovate[bot])
• docs: Add Bottlerocket OS to validated distros (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR docs: Add Bottlerocket OS to validated distros #25390, @nebril)
• docs: document missing entity 'ingress' (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR docs: document missing entity 'ingress' #25665, @mhofstetter)
• docs: Fix broken link to backends leak issue (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR docs: Fix broken link to backends leak issue #25278, @akhilles)
• docs: Improve BGP Control Plane page (Backport PR v1.13 backports 2023-05-28 #25731, Upstream PR docs: Improve BGP Control Plane page #23939, @krouma)
• gateway-api: Remove unused function check (gateway-api: Remove unused function check #26058, @ferozsalam)
• install: Fail helm if kube-proxy-replacement is not valid (Backport PR v1.13 Backports 2023-06-07 #25977, Upstream PR install: Fail helm if kube-proxy-replacement is not valid #25907, @jrajahalme)
• ipsec: Fix cleanup of XFRM states and policies (Backport PR v1.13 backports 2023-06-09 #26079, Upstream PR ipsec: Fix cleanup of XFRM states and policies #26072, @pchaigno)
• Slim down Node handler interface (Backport PR v1.13 backports 2023-06-02 #25855, Upstream PR Slim down Node handler interface #25450, @bimmlerd)
• Slim down Node handler interface (Backport PR [v1.13] Author backports of Node Handler for wireguard, and dependent PR #25923, Upstream PR Slim down Node handler interface #25450, @bimmlerd)
• test/provision/compile.sh: Make usable from dev VM (Backport PR v1.13 Backports 2023-05-16 #25503, Upstream PR test/provision/compile.sh: Make usable from dev VM #25352, @jrajahalme)
• Update network attacker sections of the threat model (Backport PR v1.13 Backports 2023-06-07 #25977, Upstream PR Update threat model #25640, @ferozsalam)

Other Changes:

• envoy: Bump envoy version to v1.23.10 (envoy: Bump envoy version to v1.23.10 #25884, @mhofstetter)
• install: Update image digests for v1.13.3 (install: Update image digests for v1.13.3 #25726, @thorn3r)
• wireguard: Always unset fwMark (wireguard: Always unset fwMark #25858, @brb)

[qmonnet]

Looks all good, thank you!

[ti-mo]

Sorry, but we need to block this release on #26091. #25795 dropped the >> scaler shift at the end of GetJtime(). This is bad, this value is used for CT GC. I'll work on a fix asap.

[ti-mo]

This PR will need to be included: #26197.

[michi-covalent]

some PRs got merged. let's re-generate release notes. closing ✅