Release Test Plan 1.4.4

Issues targeted for this release - List of Github issues

Contributor is testing: Chris, Yang, Roger, Nitendra, Khushboo

Testing Version: v1.4.4-rc1, v1.4.4-rc2

e2e Pipelines

✅ - Analyze test result of SLES on 1.4.x (AMD) - Nitendra

Build 334 : 2 failed

tests.test_basic.test_aws_iam_role_arn failed on SLES on 1.4.x (AMD) #6912

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

Build 348 (RC2) : 1 fail

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

✅ - Analyze test result of SLES on 1.4.x (ARM) - Nitendra

Build 324 : 2 failed

tests.test_basic.test_default_storage_class_syncup failed on SLES on 1.4.x (ARM) #6913

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

Build 336 (RC2) : 1 fail

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

✅- Analyze upgrade test result of SLES on 1.4.x (AMD) - Chris

Build 335 : 1 fail

Build 344 (RC2) : 1 fail

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

✅ - Analyze upgrade test result of SLES on 1.4.x (ARM) - Chris

Build 328 : 1 fail

Build 337 (RC2) : 1 fail

test_csi_umount_when_longhorn_block_device_is_disconnected_unexpectedly #6533

Non-e2e Pipelines

✅ - Benchmark (compare with 1.4.3) - Chris

RC1 ✅ RC2 ✅

✅ - Air-Gap (manifest, helm chart) - Roger

RC1 - manifest, chart ✅

RC2 - manifest, chart ✅

✅ - Vulnerability scanning - Nitendra

Build 1181 : ✅ 66 HIGH, 0 CRITICAL v1.4.4.-rc1

Build 1190 : ✅ 69 HIGH, 0 CRITICAL v1.4.4.-rc2

✅ - Negative Testing - Yang

• Reboot Volume Node While Workload Heavy Writing

RC1 ✅

RC2 ✅

• Power Off Volume Node For More Than Pod Eviction Timeout While Workload Heavy Writing

RC1 ✅

RC2 ✅

• Reboot Volume Node While Replica Rebuilding

RC1 ✅

RC2 ✅

• Reboot Replica Node While Replica Rebuilding

RC1 ✅

RC2 ✅

• Restart Cluster While Workload Heavy Writing

RC1 ❌

RC2 ✅

https://github.com/longhorn/longhorn/issues/6924

• Reboot Volume Node While Heavy Writing And Recurring Jobs Exist

This isn't going to work since the corresponding PR hasn't merged yet: https://github.com/longhorn/longhorn-tests/pull/1539

Manual Tests

✅ - Reboot Volume node while RWX workload heavy writing - Yang

RC2 ✅

✅ - Pull backup created by another Longhorn system - Yang

RC1 ✅

RC2 ✅

✅ - Replica Rebuilding - Roger

RC1 ✅

RC2 ✅

✅ - Node drain and deletion test - Roger

RC1 ✅

RC2 ✅

✅ - Single replica node down - Chris

RC1 ✅

✅❌ - Uninstallation Checks - Nitendra

6895

✅ - Kubernetes upgrade test - Chris

from v1.27.3+k3s1 to v1.28.2+k3s1, from v1.25.10+rke2r1 to v1.26.8+rke2r1 - RC1 ✅ RC2 ✅

✅ - Longhorn upgrade test - Nitendra

RC1 ✅ RC2 ✅

✅ - Test Longhorn components recovery - Chris

RC1 ✅ RC2 ✅

✅ - Test Backing Image during Longhorn upgrade - Roger

RC1 ✅

RC2 ✅

✅ - Test Engine Crash During Live Upgrade - Nitendra

RC1 ✅ RC2 ✅

✅ - Test system upgrade with a new storage class being default - Chris

RC1 ✅ RC2 ✅

Chart Testing

✅ - v2.7.x Rancher - Khushboo

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.3.3
   4. Upgrade from v1.4.3

✅ - v2.8.x Rancher - Roger

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.4.3

✅ - Longhorn Chart - Chris

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to greater than 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.4.3
   4. Upgrade from v1.4.4

RC2 ✅