StatefulSets with volumeClaimTemplates trigger constant updates

[vmax]

If a hook returns StatefulSet containing volumeClaimTemplates, Metacontroller is constantly trying to patch the object (despite the hook returning the same object).

Relevant log:

{"level":"Level(-4)","ts":1658169024.02068,"logger":"composite.hello-controller","msg":"Sync","object":{"name":"your-name","namespace":"default"}}
{"level":"Level(-5)","ts":1658169024.0232823,"msg":"Diff between observed and desired","mergePatch":{"spec":{"volumeClaimTemplates":[{"metadata":{"name":"www"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"my-storage-class"}}]}}}
{"level":"info","ts":1658169024.0233111,"msg":"Updating","parent":{"apiVersion":"example.com/v1","kind":"HelloWorld","namespace":"default","name":"your-name"},"child":{"apiVersion":"apps/v1","kind":"StatefulSet","name":"web"},"reason":"InPlace update strategy selected"}

Relevant object:

{
  "apiVersion": "apps/v1",
  "kind": "StatefulSet",
  "metadata": {
    "name": "web"
  },
  "spec": {
    "selector": {
      "matchLabels": {
        "app": "nginx"
      }
    },
    "serviceName": "nginx",
    "replicas": 3,
    "minReadySeconds": 10,
    "template": {
      "metadata": {
        "labels": {
          "app": "nginx"
        }
      },
      "spec": {
        "terminationGracePeriodSeconds": 10,
        "containers": [
          {
            "name": "nginx",
            "image": "k8s.gcr.io/nginx-slim:0.8",
            "ports": [
              {
                "containerPort": 80,
                "name": "web"
              }
            ],
            "volumeMounts": [
              {
                "name": "www",
                "mountPath": "/usr/share/nginx/html"
              }
            ]
          }
        ]
      }
    },
    "volumeClaimTemplates": [
      {
        "metadata": {
          "name": "www"
        },
        "spec": {
          "accessModes": [
            "ReadWriteOnce"
          ],
          "volumeMode": "Filesystem",
          "storageClassName": "my-storage-class",
          "resources": {
            "requests": {
              "storage": "1Gi"
            }
          }
        }
      }
    ]
  }
}

Relevant bit of observed state:

"`volumeClaimTemplates`)": [
{
  "apiVersion": "v1",
  "kind": "PersistentVolumeClaim",
  "metadata": {
    "creationTimestamp": null,
    "name": "www"
  },
  "spec": {
    "accessModes": [
      "ReadWriteOnce"
    ],
    "resources": {
      "requests": {
        "storage": "1Gi"
      }
    },
    "storageClassName": "my-storage-class",
    "volumeMode": "Filesystem"
  },
  "status": {
    "phase": "Pending"
  }
}
]

Relevant differences are .spec.volumeClaimTemplates[].status ({"phase": "Pending"}) and .spec.volumeClaimTemplates[].metadata.creationTimestamp (null)

Looks like this happens because status and metadata.creationTimestamp are only cleared from the top-level object (StatefulSet) but not child object (PersistentVolumeClaim inside of volumeClaimTemplates).

[grzesuav]

hmm metacontroller does not try to remove fields not managed by it. Maybe it is because volumeClaimTemplates is an array, need to dig a bit

[ShiroDN]

It seems I have the same issue.

Only difference between observed and desired state is "creationTimestamp": null

log:

{"level":"Level(-5)","ts":1659517976.6638033,"msg":"Diff between observed and desired","mergePatch":{"spec":{"dataVolumeTemplates":[{"apiVersion":"cdi.kubevirt.io/v1beta1","kind":"DataVolume","metadata":{"name":"test-1"},"spec":{"pvc":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"30Gi"}}},"source":{"pvc":{"name":"linux-image","namespace":"virt-images"}}}},{"metadata":{"name":"cloudinit-test-1"},"spec":{"pvc":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"2Mi"}}},"source":{"http":{"url":"https://example.com"}}}}]}}}

observed:

"dataVolumeTemplates": [
  {
    "apiVersion": "cdi.kubevirt.io/v1beta1",
    "kind": "DataVolume",
    "metadata": {
      "creationTimestamp": null,
      "name": "test-1"
    },
    "spec": {
      "pvc": {
        "accessModes": [
          "ReadWriteOnce"
        ],
        "resources": {
          "requests": {
            "storage": "30Gi"
          }
        }
      },
      "source": {
        "pvc": {
          "name": "linux-image",
          "namespace": "virt-images"
        }
      }
    }
  },
  {
    "metadata": {
      "creationTimestamp": null,
      "name": "cloudinit-test-1"
    },
    "spec": {
      "pvc": {
        "accessModes": [
          "ReadWriteOnce"
        ],
        "resources": {
          "requests": {
            "storage": "2Mi"
          }
        }
      },
      "source": {
        "http": {
          "url": "https://example.com"
        }
      }
    }
  }
],

desired:

"dataVolumeTemplates": [
  {
    "apiVersion": "cdi.kubevirt.io/v1beta1",
    "kind": "DataVolume",
    "metadata": {
      "name": "test-1"
    },
    "spec": {
      "pvc": {
        "accessModes": [
          "ReadWriteOnce"
        ],
        "resources": {
          "requests": {
            "storage": "30Gi"
          }
        }
      },
      "source": {
        "pvc": {
          "name": "linux-image",
          "namespace": "virt-images"
        }
      }
    }
  },
  {
    "metadata": {
      "name": "cloudinit-test-1"
    },
    "spec": {
      "pvc": {
        "accessModes": [
          "ReadWriteOnce"
        ],
        "resources": {
          "requests": {
            "storage": "2Mi"
          }
        }
      },
      "source": {
        "http": {
          "url": "https://example.com"
        }
      }
    }
  }
],

[grzesuav]

@vmax do you have an example of your CRD's and controller ? As it seems similar to catset example, which is working.

I will try to setup example on my own but it can take some time

[vmax]

@grzesuav sorry for the late reply!

CRD:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: helloworlds.example.com
spec:
  group: example.com
  scope: Namespaced
  names:
    kind: HelloWorld
    plural: helloworlds
    singular: helloworld
  versions:
  - name: v1
    served: true
    storage: true
    subresources:
      status: {}
    schema:
      openAPIV3Schema:
        type: object

Controller:

apiVersion: metacontroller.k8s.io/v1alpha1
kind: CompositeController
metadata:
  name: hello-controller
spec:
  generateSelector: true
  parentResource:
    apiVersion: example.com/v1
    resource: helloworlds
  resyncPeriodSeconds: 5
  childResources:
  - apiVersion: apps/v1
    resource: statefulsets
    updateStrategy:
      method: InPlace
  hooks:
    sync:
      webhook:
        url: http://hello-controller/sync