Retain the OCP cluster only if ocp-e2e-run fails

[leochr]

The OCP cluster is retained when kind-e2e-run (VM tests) fails. The OCP cluster must be freed up if ocp-e2e-run passed:

[FAILED] kind-e2e-run: Exited (2) 30 minutes ago
[PASSED] ocp-e2e-run
Your acceptance test failed, the cluster will be retained for 6 hours.  If you need more time to debug ( 72 hours ):
debug of cluster may be required, issue @ebc debug wlo_1678285570 in #was-ebc channel to keep cluster for debug
issue @ebc debugcomplete wlo_1678285570 when done debugging in #was-ebc channel 
access console at: https://console-openshift-console.apps.xyz.cp.fyre.ibm.com
credentials: kubeadmin/xyz

[leochr]

@dtadcox could you please look into this? The OCP cluster is unnecessarily retained in some cases and could result in other builds being unable to get an OCP cluster

[dtadcox]

@leochr To be clear, you want to retain OCP cluster if ocp-e2e-run fails and ONLY if that run fails. We DO NOT want to retain the OCP cluster when the kind-e2e-run fails? Currently, both tests are launched via the acceptance-test.sh script. It will return a failure code if either test fails, currently. So the solution would be to either break the two test launch scripts into separate scripts. Problem with doing that is you would lose the ability to run them simultaneously. Not sure that is a big deal or not. I suppose another option would be to pass results back via a file, or some other means to the calling script. That would need some prototyping.

[leochr]

@dtadcox Correct, we should retain the OCP cluster only if ocp-e2e-run failed (on that particular architecture).

We could separate kind-e2e-run, but still somehow run them in parallel.

The tests for X, P and Z run in sequence right now. But @yongja79 is looking into running them in parallel. I'll book a meeting to discuss this further. Thanks

[dtadcox]

pr: #529

[dtadcox]

Work is complete and delivered