Compliance operator scans may leave namespaces stuck terminating

[rtheis]

The Red Hat OpenShift on IBM Cloud test pipeline uses the compliance operator for compliance scanning. Most of the time, this testing goes smoothly. However, there are times when there are general compliance operator scan problems that leave profilebundle and/or compliancescan resources behind. These resources have a Kubernetes finalizer that is not cleared and thus namespace termination is blocked. We have tried to order compliance operator cleanup but the problem keeps occurring.

Issue transferred from openshift#831.

[jhrozek]

(Just copying what I proposed in the opennshift issue, hoping someone else might have a better opinion)
I'm afraid I don't have a great solution except prune the profile bundles manually. The order in which I'd do that would be:

oc delete ssb --all -n$namespace
# in case you're using raw suites/scans for some reason
oc delete suites --all -n$namespace
oc delete scans --all -n$namespace
oc delete pb.compliance --all -n$namespace

Maybe we should document this more prominently? We do have https://github.com/ComplianceAsCode/compliance-operator/blob/master/doc/install.md#namespace-removal which links to the Makefile tear-down targets, but I'm not sure (read: haven't tested) if removing the CRD removes the objects.

[rtheis]

When we moved to OCP version 4.12, we started to see the same problem for compliancesuites resources too. This looks very similar to #231.