EPIC: Optimized Version of Compact Blocks

[evan-forbes]

We have the base compact blocks implementation in #1191. This implementation nails the basics of what compact blocks are supposed to be. Validators can not only avoid gossiping block data twice by gossiping txs in the mempool, but they can also gossip txs before the actual proposal period.

The first iteration was not intended to be final by any means, and we still have some work to do before it can become a viable strategy for increasing consensus throughput. The main current blocker is that gossiping txs before and during the proposal period is far too slow.

CAT is optimized for efficiency, and not for distribution speed. Not only are there extra hops required for the have and want peer state messages, but there is no way to break up the blob into smaller pieces, both of which dramatically increase the time required to distribute block data. CAT also does not yet have any mechanism to prioritize txs that are included in a proposal relative to those that are simply being propagated in the mempool. This results in very low consensus throughput, since proposers often include txs that are not widely distributed, and then have no mechanism to distribute them quickly after the proposal has been reached.

To address these issues we can (in order):

• Only reap txs that have been in the proposer’s mempool for a configurable amount of time. #1335
• Add a high priority proposal txs channel in CAT #1336
• Add more peers only to the mempool reactor #1337
• Ask for wants from the first peer that gossips the compact block #1338
• Distribute block data in compact blocks as block parts #1339