Release plan / Release 1.25.5 / 1.27.2

[rborundia]

Hi,
Could you please let us know when are we going to get release for build number 1.27.2/1.25.5 ?
We are looking for following fix(PR #7129). Release1.27.1 and 1.25.4 does not have the following fix. We are running to crio panic with double close of channel when we delete a deployment and crio's container stop function goes into retry.

commit 6dec88e
Author: Peter Hunt pehunt@redhat.com
Date: Thu Jul 6 16:58:49 2023 -0400

oci: simplify stopping code

Before, there was the possibility load could cause cri-o to segfault from double closing of channels.

this PR aims to simplify container stop code while retaining the required behavior.

Now, the first stop begins a registration process where the container stop begins and new timeouts
come in to interrupt. There are two commuincation channels, and only one location where they can be closed.
This also adds a watcher mechanism so callers can wait on the container stop

Signed-off-by: Peter Hunt pehunt@redhat.com

Panic:
Oct 25 02:18:41 dssserv8 crio[2263]: panic: close of closed channel
Oct 25 02:18:41 dssserv8 crio[2263]: goroutine 2601932 [running]:
Oct 25 02:18:41 dssserv8 crio[2263]: panic({0x5641fb169a00, 0x5641fb420080})
Oct 25 02:18:41 dssserv8 crio[2263]: /usr/lib64/go-1.19/src/runtime/panic.go:987 +0x3ba fp=0xc0020f3990 sp=0xc0020f38d0 pc=0x5641f92f017a
Oct 25 02:18:41 dssserv8 crio[2263]: runtime.closechan(0xc0018cbf20)
Oct 25 02:18:41 dssserv8 crio[2263]: /usr/lib64/go-1.19/src/runtime/chan.go:365 +0x3fc fp=0xc0020f39d0 sp=0xc0020f3990 pc=0x5641f92bae5c
Oct 25 02:18:41 dssserv8 crio[2263]: github.com/cri-o/cri-o/internal/oci.WaitContainerStop({0x5641fb439d48?, 0xc0008b72c0?}, 0xc001402580, 0x1bf08eb000, 0>
Oct 25 02:18:41 dssserv8 crio[2263]: /home/abuild/rpmbuild/BUILD/cri-o-1.27.1/internal/oci/runtime_oci.go:864 +0x671 fp=0xc0020f3b18 sp=0xc0020f39>
Oct 25 02:18:41 dssserv8 crio[2263]: github.com/cri-o/cri-o/internal/oci.(*runtimeOCI).StopContainer(0xc0006b9170?, {0x5641fb439d48?, 0xc0008b7290?}, 0xc0>
Oct 25 02:18:41 dssserv8 crio[2263]: /home/abuild/rpmbuild/BUILD/cri-o-1.27.1/internal/oci/runtime_oci.go:920 +0x585 fp=0xc0020f3d10 sp=0xc0020f3b>
Oct 25 02:18:41 dssserv8 crio[2263]: github.com/cri-o/cri-o/internal/oci.(*Runtime).StopContainer(0x5641fb439d48?, {0x5641fb439d48?, 0xc0008b71a0?}, 0x0?,>
Oct 25 02:18:41 dssserv8 crio[2263]: /home/abuild/rpmbuild/BUILD/cri-o-1.27.1/internal/oci/oci.go:284 +0x11c fp=0xc0020f3da8 sp=0xc0020f3d10 pc=0x>
Oct 25 02:18:41 dssserv8 crio[2263]: github.com/cri-o/cri-o/server.(*Server).stopContainer(0xc0000fbc00, {0x5641fb439d48?, 0xc001b9c4e0?}, 0xc001402580, 0>
Oct 25 02:18:41 dssserv8 crio[2263]: /home/abuild/rpmbuild/BUILD/cri-o-1.27.1/server/container_stop.go:60 +0x43b fp=0xc0020f3f08 sp=0xc0020f3da8 p>
Oct 25 02:18:41 dssserv8 crio[2263]: github.com/cri-o/cri-o/server.(*Server).stopPodSandbox.func1()

[kwilczynski]

@rborundia, thank you for asking!

Work on the release 1.27.2 is already ongoing per #7430.

However, I wonder if we have plans to release a new 1.25.x version at this time.

That said, regardless of which release branch, we would need to ensure this bug fix is backported.

@saschagrunert and @haircommander, your thoughts?

[haircommander]

I have just attempted to cut 1.26.5, 1.25.5 artifacts are broken so we may need to do a 1.25.6 in the new year. sorry for the delay. 1.27.2 should be ready now

[rborundia]

@haircommander
I see that 1.27.2 is released. Thank you for the release. Do you know when binaries will be available at https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.27:/1.27.2/CentOS_8/

[kwilczynski]

@rborundia, I might need to be corrected about this concerning older releases. However, the packages repository has moved.

Check the following blog post:

• CRI-O is moving towards pkgs.k8s.io

More details can be found at cri-o/packaging project.

[haircommander]

sorry @rborundia I messed one last thing up. Hopefully it works now. @kwilczynski you were correct about needing to be corrected, we'll remain on the old paths for 1.28 and older :)

[kwilczynski]

@haircommander, no worries. I am happy to be wrong about this! 😄