[WIP] Cluster mempool implementation #28676
Commits on May 8, 2024
-
-
-
-
-
-
Limit mempool size based on chunk feerate
Rather than evicting the transactions with the lowest descendant feerate, instead evict transactions that have the lowest chunk feerate. Once mining is implemented based on choosing transactions with highest chunk feerate (see next commit), mining and eviction will be opposites, so that we will evict the transactions that would be mined last.
-
-
-
-
policy: Remove CPFP carveout rule
The addition of a cluster size limit makes the CPFP carveout rule useless, because carveout cannot be used to bypass the cluster size limit. Remove this policy rule and update tests to no longer rely on the behavior.
-
-
Implement new RBF logic for cluster mempool
With a total ordering on mempool transactions, we are now able to calculate a transaction's mining score at all times. Use this to improve the RBF logic: - we no longer enforce a "no new unconfirmed parents" rule - we now require that the mempool's feerate diagram must improve in order to accept a replacement TODO: update functional test feature_rbf.py to cover all our new scenarios.
-
-
-
-
Use cluster linearization for transaction relay sort order
Previously, transaction batches were first sorted by ancestor count and then feerate, to ensure transactions are announced in a topologically valid order, while prioritizing higher feerate transactions. Ancestor count is a crude topological sort criteria, so replace this with linearization order so that the highest feerate transactions (as would be observed by the mining algorithm) are relayed before lower feerate ones, in a topologically valid way. This also fixes a test that only worked due to the ancestor-count-based sort order.
-
Remove CTxMemPool::GetSortedDepthAndScore
The mempool clusters and linearization permit sorting the mempool topologically without making use of ancestor counts.
-
Reimplement GetTransactionAncestry() to not rely on cached data
In preparation for removing ancestor data from CTxMemPoolEntry, recalculate the ancestor statistics on demand wherever needed.
-
-
Stop enforcing ancestor size/count limits
The cluster limits should be sufficient.
-
-
Use mempool/txgraph to determine if a tx has descendants
Remove a reference to GetCountWithDescendants() in preparation for removing this function and the associated cached state from the mempool.
-
Calculate descendant information for mempool RPC output on-the-fly
This is in preparation for removing the cached descendant state from the mempool.
-
test: fix rbf carveout test in mempool_limit.py
Minimal fix to the test that the RBF carveout doesn't apply in certain package validation cases. Now that RBF carveout doesn't exist, we can just test that the cluster count limit is respected (in preparation for removing the descendant limit altogether).
-
Stop enforcing descendant size/count limits
Cluster size limits should be enough.
-
wallet: Replace max descendantsize with clustersize
With the descendant size limits removed, replace the concept of "max number of descendants of any ancestor of a given tx" with the cluster count of the cluster that the transaction belongs to.
-
-
Eliminate RBF workaround for CPFP carveout transactions
The new cluster mempool RBF rules take into account clusters sizes exactly, so with the removal of descendant count enforcement this idea is obsolete.
-
-
-
-
-
-
-
Rework removeForBlock so that clusters are only touched once
Also remove extra linearization that was happening and some logging
-
Simplify ancestor calculation functions
Now that ancestor calculation never fails (due to ancestor/descendant limits being eliminated), we can eliminate the error handling from CalculateMemPoolAncestors.
-
-
-
-
-
-
-
Switch to using the faster CalculateDescendants
The only place we still use the older interface is in policy/rbf.cpp, where it's helpful to incrementally calculate descendants to avoid calculating too many at once (or cluttering the CalculateDescendants interface with a calculation limit).
-
-
-
-
-
-
-
-
-
-
Eliminate need for ancestors in PackageV3Checks
TO DO: Rewrite unit tests for PV3C to not lie about mempool parents, so that we can push down the parent calculation into v3_policy from validation.
-
-
-
-
bench: add more mempool benchmarks
Add benchmarks for: - mempool update time when blocks are found - adding a transaction - performing the mempool's RBF calculation - calculating mempool ancestors/descendants
-
fuzz: try to add more code coverage for mempool fuzzing
Including test coverage for mempool eviction and expiry
-
-
-
-
fuzz: remove comparison between mini_miner block construction and miner
This is in preparation for eliminating the block template building happening in mini_miner, in favor of directly using the linearizations done in the mempool.
