Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Optimize] Skip proposing duplicated transmissions #3178

Merged
merged 9 commits into from Mar 22, 2024

Conversation

raychu86
Copy link
Contributor

@raychu86 raychu86 commented Mar 21, 2024

Motivation

This PR skips adding transmissions to proposals if the transmission was included in storage. This is because if the node's proposal/certificate gets included in a subdag, it must also include the previous certificates, meaning we'd have duplicated transmission ids.

This optimization reduces state bloat in blocks and also opens more space in proposals for unique transmissions, which would allow more TPS. The second change in this PR processes more transmissions from the worker in case there were some in the initial num_transmissions_per_worker that were skipped.

This PR also increase the number of unconfirmed transmissions we pass into the memory pool. Previously it was bounded by MAX_TRANSMISSIONS_PER_BATCH, however if any of these transmissions are invalid or duplicated, then we effectively limit the theoretical throughput. We still only include MAX_TRANSMISSIONS_PER_BATCH in each proposal, however we now have a larger pool of transmissions to choose from.

TLDR - The changes are:

  1. Skip transmissions that are already included in storage.
  2. Increase number of transmissions sent into the mempool.

This PR needs to be validated with internal devnet testing, particularly with the mempool change.

Test Plan

A test has been added that we properly skip the transmissions that already exist in storage.

@raychu86 raychu86 requested review from kpandl and vicsn March 21, 2024 01:18
Copy link
Collaborator

@vicsn vicsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic LGTM, makes a lot of sense

@raychu86 raychu86 marked this pull request as ready for review March 21, 2024 17:47
@kpandl
Copy link
Contributor

kpandl commented Mar 22, 2024

Did an initial run and was able to reach around 38 TPS on an ongoing basis with credits.aleo transfer_public transactions

@howardwu howardwu merged commit 03e7928 into mainnet-staging Mar 22, 2024
@howardwu howardwu deleted the unique-transmissions branch March 22, 2024 19:26
@howardwu howardwu changed the title Skip proposing duplicated transmissions [Optimize] Skip proposing duplicated transmissions Mar 23, 2024
@howardwu
Copy link
Member

@onetrickwolf's testing has also confirmed that block times have fallen significantly with this PR, when there are tx-cannons driving the network forward

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants