Release Test Plan 1.4.2

Issues targeted for this release - List of Github issues

QAs: Chris, Yang, Roger, Khushboo, Ray

Testing Version: v1.4.2-rc1

e2e Pipelines

✅ - Analyze test result of SLES on 1.4.x (AMD) - Chris

v1.4.2-rc1: 3 failures

✅ - Analyze test result of SLES on 1.4.x (ARM) - Chris

v1.4.2-rc1: 2 failures

✅ - Analyze upgrade test result of SLES on 1.4.x (AMD) - Yang

v1.4.2-rc1: 4 failures

✅ - Analyze upgrade test result of SLES on 1.4.x (ARM) - Yang

v1.4.2-rc1: 2 failures

✅ - Analyze test result of SLE Micro on 1.4.x (AMD) - Roger

v1.4.2-rc1: 2 failures.

✅ - Analyze test result of SLE Micro on 1.4.x (ARM) - Roger

v1.4.2-rc1: 4 failures.

✅ - Analyze two stage test result of SLES on 1.4.x (AMD) 1.2.6 → 1.3.2 → 1.4.2-rc - Chris

v1.4.2-rc1: 5 failures

✅ - Analyze two stage test result of SLES on 1.4.x (ARM) 1.2.6 → 1.3.2 → 1.4.2-rc - Chris

v1.4.2-rc1: 6 failures

Non-e2e Pipelines

✅ - Benchmark (compare with 1.4.1) - Yang

✅ - Air-Gap (manifest, helm chart, rancher chart) - Roger

✅ RC1

✅ - Vulnerability scanning - Yang

v1.4.2-rc1: 70 HIGH, 0 CRITICAL

Manual Testing

HA:

✅ - Replica Rebuilding - Roger

✅ RC1

Node not ready:

✅ - Entire cluster down - Chris

After entire cluster down(include control plane), power up all nodes wait them ready, deployment data intact, e2e core test cases all passed.

✅ - Test kubelet restart on a node of the cluster - Yang

✅ - Node disconnection test - Roger

✅ RC1

✅ - The node the restore volume attached to is down - Chris

✅ - Node drain and deletion test - Yang

✅ - Physical node down - Roger

Need to update test case https://github.com/longhorn/longhorn/issues/5900.

✅ - Single replica node down - Chris

✅ - Physical node reboot - Yang

Resiliency:

✅ - Test Longhorn components recovery - Roger

✅ - Uninstallation Checks - Chris

Upgrade:

✅ - Test Engine Crash During Live Upgrade - Yang

✅ - Kubernetes upgrade test - Roger

✅ - Longhorn upgrade test - Chris

Chart Testing

❌ - v2.6.x Rancher - Not required Rancher v2.6.x is EOM

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to greater than 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.3.3
   4. Upgrade from v1.4.1

✅ - v2.7.x Rancher - Khushboo

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.3.3
   4. Upgrade from v1.4.1

✅ - Longhorn Chart - Roger

✅ RC1

   Prereq: Set Concurrent Automatic Engine Upgrade Per Node Limit to greater than 0
   Test steps:
   1. Fresh installation
   2. Uninstallation
   3. Upgrade from v1.3.3
   4. Upgrade from v1.4.1

RobotFramework

Node not ready

• Kubelet restart
  • 
  • 
• Reboot
• Powering off

---