[BUG] pod is in the cache, cannot be assumed KB 083beta19 + K8S 1.21.5

[shanshanying]

1. KB 0.8.3-beta19
2. K8s version: 1.21.5
3. create a PG cluster

[shanshanying]

• [IMPROVEMENT] [CloneSet] [Lifecycle] Kruise v1.4+ works not so well with kube-scheduler <= v1.21 openkruise/kruise#1485
• Scheduler fails to schedule due to cyclic errors on pod forget and assume kubernetes/kubernetes#101877
• Scheduler PriorityQueue Update function  kubernetes/kubernetes#98828

[ahjing99]

Cannot recreate with both newly installed and upgrade kb, thus I think this is a k8s env problem

newly install 0.8.3 on k3s 1.21.14
https://github.com/apecloud/kubeblocks/actions/runs/9077390363

Upgrade from 0.8.2 to 0.8.3 on on k3s 1.21.14
https://github.com/apecloud/kubeblocks/actions/runs/9077373907

[shanshanying]

reproduced on k8s 1.21.1

With some anoying warnings at the very beginning, but the pod got scheduled seconds later.

And this is a known issues for k8s <=1.21.
The cause explained in OpenKruise is

when a pod is created, the scheduler will watch and enqueue it to the scheduling queue. After the scheduling algorithm is completed (finished finding a node), BIND will be performed concurrently. Only when BIND succeeds will the POD be considered scheduled. If someone modifies the pod before the whole process is completed, the POD will be requeued. ...... the kube-scheduler will not skipPODUpdate and will retry to schedule, this will cause continuous annoying ERROR log and Warning events (but will not actually affect POD scheduling).