Change MAX_EFFECTIVE_BALANCE will cause prysm or lighthouse failed to sync with lodestar

[canximan]

Describe the bug

I tried to run a new testnet, with MAX_EFFECTIVE_BALANCE = 320 ETH.
I run the testnet with 1 EC: geth, 3 consensus nodes: 2 - lodestar nodes + 1 lodestar validator, 1 prysm node, 1 lighthouse node.

Lodestar can create new block and successfully past the merge. But both prysm and lighthouse failed to sync with loadestar.

Same thing happen when I start another testnet: 1 lighthouse node + 1 lighthouse validator, 1 prysm node, 1 lodestar node.
Pysm can sync with ligthouse, by lodestar was not.

Error log from Prysm:

[2024-04-02 14:57:55]  WARN initial-sync: Block processing failure error=failed to validate consensus state transition function: could not execute state transition: could not process block: could not process block header: proposer index: 59 is different than calculated: 90 firstSlot=6 firstUnprocessed=6 lastSlot=6 root=0x965564714926063dad38b88ec0309a8fdc1148ca3708bf87e0a0efaa9c3dc111

Error log from lighthouse:

Apr 02 07:57:49.703 DEBG Missed snapshot cache                   block_delay: None, parent_root: 0x630339bc76ca60d3bc43829776777cef9823754f41ae0aa432fde04d13ba344d, parent_slot: 0, slot: 6, service: beacon
Apr 02 07:57:49.703 DEBG Invalid block received                  outcome: IncorrectBlockProposer { block: 59, local_shuffling: 90 }, msg: peer sent invalid block
Apr 02 07:57:49.703 DEBG Batch processing failed                 service: sync, error: Peer sent invalid block. Reason: IncorrectBlockProposer { block: 59, local_shuffling: 90 }, imported_blocks: 0, last_block_slot: 28, chain: 8800787327743628209, first_block_slot: 0, batch_epoch: 0
Apr 02 07:57:49.704 DEBG Batch processing result                 client: Lodestar: version: v1.16.0, batch_epoch: 0, start_slot: 0, end_slot: 31, downloaded: 0, processed: 0, processed_no_penalty: 27, state: Processing(16Uiu2HAmSdjN4jTXvkGbYxbaL2ko8PezhsZAMVQR5nW8Hgiit7VC), batch_ty: blocks, result: NonFaultyFailure, chain: 8800787327743628209, service: sync

I don't know if this is lodestar's bug or tech debt.

Expected behavior

Both prysm and lighthouse can sync with lodestar when start new testnet with MAX_EFFECTIVE_BALANCE = 320 ETH.

Steps to reproduce

1. Set MAX_EFFECTIVE_BALANCE = 320 ETH
2. Start lodestar node + lodestar validator.
3. Run prysm and lighthouse with same config and connect to lodestar.

Additional context

No response

Operating system

macOS

Lodestar version or commit hash

v1.16.0