Release Test Plan 1.5.1

Issues targeted for this release - List of Github issues

Contributor is testing: Chris, Yang, Roger, Khushboo

Testing Version: v1.5.1-rc1

e2e Pipelines

✅ - Analyze test result of SLES on 1.5.x (AMD) - Roger

v1.5.1-rc1: 4 failures.

✅ - Analyze test result of SLES on 1.5.x (ARM) - Roger

v1.5.1-rc1: 3 failures.

✅ - Analyze upgrade test result of SLES on 1.5.x (AMD) - Yang

v1.5.1-rc1: 2 failures

failed test cases are either passed after rerun or known issues have tickets associated with them. for more details: https://github.com/longhorn/longhorn/issues/6274#issuecomment-1639174390

✅ - Analyze upgrade test result of SLES on 1.5.x (ARM) - Yang

v1.5.1-rc1: 3 failures

failed test cases are either passed after rerun or known issues have tickets associated with them. for more details: https://github.com/longhorn/longhorn/issues/6274#issuecomment-1639174390

✅ - Analyze two stage test result of SLES on 1.5.x (AMD) 1.3.3 → 1.4.3 → 1.5.0-rc - Chris

rc1

test_csi_snapshotter.py rerun : 0 failure

test_settings.py rerun : 0 failure

test_inc_restoration_with_multiple_rebuild_and_expansion known issue, #5496

✅ - Analyze two stage test result of SLES on 1.5.x (ARM) 1.3.3 → 1.4.3 → 1.5.0-rc - Chris

rc1

test_basic.py rerun : 0 failure

test_csi_snapshotter.py rerun : 0 failure

test_inc_restoration_with_multiple_rebuild_and_expansion known issue, #5496

Non-e2e Pipelines

✅ - Benchmark (compare with 1.4.2) - Yang

v1.5.1-rc1 ✅

✅ - Air-Gap - Chris

rc1 - manifest, chart ✅

✅ - Vulnerability scanning - Chris

rc1- 37 failures ✅

Manual Testing

✅ - Uninstallation Checks - Roger

✅ RC1

Upgrade:

✅ - Kubernetes upgrade test - Chris

from v1.26.6+k3s1 to v1.27.3+k3s1, from v1.25.10+rke2r1 to v1.26.5+rke2r1

RC1 ✅

✅ - Longhorn upgrade test - Yang

upgrade from v1.4.3 to v1.5.1-rc1, from v1.5.0 to v1.5.1-rc1 ✅

Chart Testing

✅ - v2.7.x Rancher - Roger, Khushboo

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.4.2
   4. Upgrade from v1.4.3

✅ v2.7.5 Rancher + v1.5.1

✅ - Longhorn Chart - Roger

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to greater than 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.4.3

✅ RC1