Erlay: bandwidth-efficient transaction relay protocol #21515
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
promag
reviewed
Mar 23, 2021
naumenkogs
force-pushed
the
2021-03-erlay
branch
3 times, most recently
from
707618c
to
20ac0f4
Compare
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
This was referenced Mar 24, 2021
naumenkogs
force-pushed
the
2021-03-erlay
branch
2 times, most recently
from
d787002
to
aba98b8
Compare
naumenkogs
force-pushed
the
2021-03-erlay
branch
10 times, most recently
from
36564f5
to
af41f55
Compare
This was referenced Mar 27, 2021
This will help to reuse the code later on in the function to announce transactions.
If after decoding a reconciliation sketch it turned out to be insufficient to find set difference, request extension.
Store the initial sketches so that we are able to process extension sketch while avoiding transmitting the same data.
To be ready to respond to a sketch extension request from our peer, we should store a snapshot of our state and capacity of the initial sketch, so that we compute extension of the same size and over the exact same transactions. Transactions arriving during this reconciliation will be instead stored in the regular set.
If peer failed to reconcile based on our initial response sketch, they will ask us for a sketch extension. Store this request to respond later.
Sending an extension may allow the peer to reconcile transactions, because now the full sketch has twice as much capacity.
If a peer sent us an extension sketch, we should reconstruct a full sketch from it with the snapshot we stored initially, and attempt to decode the difference. p2p: Handle sketch extension If a peer sent us an extension sketch, we should reconstruct a full sketch from it with the snapshot we stored initially, and attempt to decode the difference.
This currently unused function is supposed to be used once a reconciliation round is done. It cleans the state corresponding to the passed reconciliation.
Once a peer tells us reconciliation is done, we should behave as follows: - if it was successful, just respond them with the transactions they asked by short ID. - if it was a full failure, respond with all local transactions from the reconciliation set snapshot - if it was a partial failure (only low or high part was failed after a bisection), respond with all transactions which were asked for by short id, and announce local txs which belong to the failed chunk.
naumenkogs
force-pushed
the
2021-03-erlay
branch
from
dcfc8b0
to
aada08e
Compare
First, extend the timeout to facilitate reconciliation relay which might take longer than previously. Second, since reconciliation can't guarantee that descendants of same-degree (from the parent) are relayed in the same order they were received, loosen the check for descendant limit. This is the case because, if the parent is flooded, the children might be both flooded and reconciled, and the decision is made independently. So even if one arrived earlier, it can be relayed later because flooding. Note, it's possible to make txreconciliation.cpp be aware of the order of adding transactions to the set, but this won't help to solve the above issue.
naumenkogs
force-pushed
the
2021-03-erlay
branch
from
aada08e
to
3e08dab
Compare
This was referenced Oct 18, 2023
This was referenced Nov 27, 2023
|
🐙 This pull request conflicts with the target branch and needs rebase. |
|
⌛ There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
|
⌛ There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Erlay Project Tracking: #28646
This is an implementation of Erlay , using primitives in the BIP-330 (see the updated spec here ). Please refer to these two to understand the design. My talk is here.
Abstract
Erlay uses both flooding (announcing using INV messages to all peers) and reconciliation to announce transactions. Flooding is expensive, so Erlay seeks to use it sparingly and in strategic locations - only well-connected publicly reachable nodes flood transactions to other publicly reachable nodes via outbound connections. Since every unreachable node is directly connected to several reachable nodes, this policy ensures that a transaction is quickly propagated to be within one hop from most of the nodes in the network.
All transactions not propagated through flooding are propagated through efficient set reconciliation. To do this, every node keeps a reconciliation set for each peer, in which transactions are placed which would have been announced using INV messages absent this protocol. Every 2 seconds every node chooses a peer from its outbound connections in a predetermined order to reconcile with, resulting in both sides learning the transactions known to the other side. After every reconciliation round, the corresponding reconciliation set is cleared.
I think both paper and the BIP motives the changes, but I’ll mention them briefly once again here:
How to review
I suggest doing
make clean && autogen.sh && configurebefore you try building it locally.