Recreating ResourceComposition instance

[devdattakulkarni]

We have observed that the following sequence of events is possible:

• Create ResourceComposition instance -> This leads to the creation of the application CRD
• KubePlus Pod gets deleted from the cluster
• ResourceComposition instance gets deleted -> This leaves behind the application CRD and its instances (since KubePlus is not around, deletion of ResourceComposition instance does not lead to delete of application CRDs and their instances)
• KubePlus is installed
• Trying to create a ResourceComposition instance to "acquire/subsume" the existing application CRD and its instances fails with the following type of error:

Error from server: error when creating "Jenkins.yaml": admission webhook "platform-as-code.crd-binding.v1" denied the request: Resource with Kind Name Jenkins exists in the cluster.

[devdattakulkarni]

@eminalparslan Can you take a look at this issue?

[devdattakulkarni]

@eminalparslan

Kubernetes finalizers will be an appropriate mechanism to handle this situation:
https://kubernetes.io/docs/concepts/overview/working-with-objects/finalizers/

The general idea will be like this:
When KubePlus Pod is about to be deleted, we add metadata.finalizers entry to all the resourcecomposition instances that are currently present in the cluster. This will prevent Kubernetes API to delete resourcecomposition instances. When KubePlus starts up, it will check for any existing resourcecomposition instances with metadata.finalizers field. If it finds such resources, KubePlus can simply clear the metadata.finalizers entry. This will cause Kubernetes API to proceed with the deletion of that resourcecomposition if one was requested while KubePlus was down. And since KubePlus is now back, the resourcecomposition deletion should naturally lead to deletion of all the children Custom Resource instances. (This assumes that KubePlus, upon startup, is able to build its internal state correctly by discovering any resourcecomposition instances that are currently present in the cluster. This functionality is present in KubePlus, though it will be good to verify it).

[devdattakulkarni]

Pre-requisites:

• Understanding of KubePlus architecture
• Understanding of Kubernetes finalizers
• Golang