Logs show: WARN frame executor queue is backlogged

[stuberman]

Checklist

• This is not a security-related bug/issue. If it is, please follow please follow the security policy.
• I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
• I am running the Latest release, the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• I did not make any code changes to lotus.

Lotus component

• lotus daemon - chain sync
• lotus fvm/fevm - Lotus FVM and FEVM interactions
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt/WinningPoSt)
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

Lotus Version

Daemon: 1.26.3+mainnet+git.560826d5c+api1.5.0
Local: lotus-miner version 1.26.3+mainnet+git.560826d5c

rustc 1.77.2 (25ef9e3d8 2024-04-09)
rustup 1.27.0 (bbb9276d2 2024-03-08)
go version go1.21.9 linux/amd64

free -h
              total        used        free      shared  buff/cache   available
Mem:          125Gi        43Gi       864Mi        10Mi        81Gi        81Gi
Swap:         199Gi        73Mi       199Gi

Repro Steps

1. Start Lotus-miner
2. See logs

Describe the Bug

After upgrading to lotus v1.26.3 from 1.25 seeing these WARN errors in miner logs continuously

Logging Information

2024-04-23T19:16:32.188Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 130, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 131, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 132, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 133, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 134, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 134, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 132, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 130, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 130, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 131, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 132, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 133, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 134, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 134, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 131, "cap": 256}
2024-04-23T19:16:32.282Z	WARN	rpc	go-jsonrpc@v0.3.1/websocket.go:676	frame executor queue is backlogged	{"queued": 129, "cap": 256}