New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Optimize] Skip proposing duplicated transmissions #3178
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic LGTM, makes a lot of sense
Did an initial run and was able to reach around 38 TPS on an ongoing basis with credits.aleo transfer_public transactions |
@onetrickwolf's testing has also confirmed that block times have fallen significantly with this PR, when there are |
Motivation
This PR skips adding transmissions to proposals if the transmission was included in storage. This is because if the node's proposal/certificate gets included in a subdag, it must also include the previous certificates, meaning we'd have duplicated transmission ids.
This optimization reduces state bloat in blocks and also opens more space in proposals for unique transmissions, which would allow more TPS. The second change in this PR processes more transmissions from the worker in case there were some in the initial
num_transmissions_per_worker
that were skipped.This PR also increase the number of unconfirmed transmissions we pass into the memory pool. Previously it was bounded by
MAX_TRANSMISSIONS_PER_BATCH
, however if any of these transmissions are invalid or duplicated, then we effectively limit the theoretical throughput. We still only includeMAX_TRANSMISSIONS_PER_BATCH
in each proposal, however we now have a larger pool of transmissions to choose from.TLDR - The changes are:
This PR needs to be validated with internal devnet testing, particularly with the mempool change.
Test Plan
A test has been added that we properly skip the transmissions that already exist in storage.