Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As Non light client: broadcast BlockHeaders #2343

Merged
merged 1 commit into from May 6, 2024

Conversation

nibhar
Copy link
Member

@nibhar nibhar commented Apr 3, 2024

This makes every non light client rebroadcast blocks they get on the block header topic. It is not ideal as it may lead to duplication depending on how pubsup messages are de-duped.
To my knowledge in this case the messages actual bytes are the distinguishing factor. If that is the case they should all be the same for micro blocks and only differ with macro blocks which have different proofs.

Resolves #2288

@nibhar nibhar added this to the Nimiq PoS Mainnet milestone Apr 3, 2024
@nibhar nibhar self-assigned this Apr 3, 2024
@nibhar nibhar force-pushed the nibhar/block-header-rebroadcast branch from 296f9a2 to 8eef5e7 Compare April 3, 2024 10:50
consensus/src/sync/live/block_queue/mod.rs Outdated Show resolved Hide resolved
consensus/src/sync/live/block_queue/mod.rs Outdated Show resolved Hide resolved
consensus/src/sync/live/block_queue/mod.rs Outdated Show resolved Hide resolved
@nibhar nibhar force-pushed the nibhar/block-header-rebroadcast branch from 8eef5e7 to fff5e2d Compare April 4, 2024 12:51
@sisou
Copy link
Member

sisou commented May 5, 2024

@nibhar how can I test if the messages are correctly de-duped with a local devnet? What log messages do I have to look for, or add?

@sisou
Copy link
Member

sisou commented May 6, 2024

Running it without any changes, I only see one log message Received block via gossipsub per block in the web client, both micro and macro blocks. But I think that message is only logged after the BlockStream already deduped the blocks? Or maybe not? I cannot find any deduping logic for BlockStream, so I think it works!

@sisou sisou force-pushed the nibhar/block-header-rebroadcast branch from fff5e2d to e62924e Compare May 6, 2024 07:50
@nibhar nibhar force-pushed the nibhar/block-header-rebroadcast branch from e62924e to 991649d Compare May 6, 2024 12:19
@nibhar
Copy link
Member Author

nibhar commented May 6, 2024

I think what should happen is that the pubsub implementation should create a unique identifier for messages. Important is that the messages we publish here are identical no matter who they are published by in terms of that identifier.
Once the pubsub implementation then starts to send the IHAVE's and IWANT's they will not be requested by any peer which already has that specific message.

So the desired behaviour would be that no traffic is produced for the additional publishes in excess of what a single publish would produce.

By the way, we do have a similar problem with macro blocks. They are assembled locally and for the asynchronous nature of the distributed system multiple nodes can produce a block at the same time, publishing it once each. However macro blocks are not unique as they can have a variety of different valid proofs. Thus they do (and should) create different identifiers and therefore will be IWANTed by other peers. Just in case this helps to see differences about this behaviour.

@jsdanielh jsdanielh force-pushed the nibhar/block-header-rebroadcast branch from 991649d to 46a5700 Compare May 6, 2024 18:01
@jsdanielh jsdanielh merged commit 46a5700 into albatross May 6, 2024
6 checks passed
@jsdanielh jsdanielh deleted the nibhar/block-header-rebroadcast branch May 6, 2024 18:35
@sisou
Copy link
Member

sisou commented May 7, 2024

Just experienced this while updating my node and it syncing up with the chain again after I rebuilt with the latest tag:

It's also broadcasting old blocks during (live?) sync.

@sisou
Copy link
Member

sisou commented May 8, 2024

I looked at the websocket connections to my peers, and I receive a message of 440B (bytes I think) every second from every full/history peer. I'm connected to 13 peers, that's 5.7kB/s. Quite a bit of traffic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Light client - not receiving all block headers over gossipsub
4 participants