The module edgecontroller message channel is full

[zhushuaifeifei]

Environment

• k8s master: v.1.21.5
• cloudcore: v1.12.1
• edgecore: v1.10.0

What happened

• First, when edgenode disconnect, metamanager still update podstatus msg to cloud, and edged still update nodestatus msg to cloud, there msg were sent to edgehub, but were not consumed. Edgehub message channel will be full, and there are lots of goroutine would be created and halt when msg channel is full.
• Second, when edgenode reconnect, lots of podstatus and nodestatus msg will be send to cloudhub immediately, all there msg would be send to edgecontroller, which will make edgecontroller message channel full. Other edge node nodestatus update msg can not be send to edgecontroller message channel in a few minutes, therefore other node will be mark notready.
• Third, even if the node is online, POD still cannot be scheduled normally and always displays Pending. However, I checked on the edge node machine. Some have been scheduled and running, and some have not been scheduled. There are still nodes online and I manually stop them. The nodes are still online and have not been updated for dozens of minutes..

What you expected to happen

• edgehub message channel not full when edge node disconnect
• edgecontroller message channel not full when edge node reconnect
• other edge node not be mark notready

How to reproduce it (as minimally and precisely as possible)

• create 100+ pod to edge node
• disconnect node for 30min
• reconnect edge node

nodestatus and podstatus update msg will be overwrite when edge node disconnect.