Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer incoming engine messages if pruner is active #8203

Open
Tracked by #8303
mattsse opened this issue May 11, 2024 · 1 comment · Fixed by #8315
Open
Tracked by #8303

Buffer incoming engine messages if pruner is active #8203

mattsse opened this issue May 11, 2024 · 1 comment · Fixed by #8315
Assignees
Labels
A-consensus Related to the consensus engine C-bug An unexpected or incorrect behavior C-enhancement New feature or request

Comments

@mattsse
Copy link
Collaborator

mattsse commented May 11, 2024

Describe the feature

currently, pruning data and engine message handling are mutually exclusive because we can't handle them concurrently due to exclusive db lock.

rn we're dropping messages in edge cases, which is not ideal.
we expect the pruner to be active for at most a few 100ms, so we should be able to delay engine message processing by buffering them in a Vecdeque and popping them in poll loop:

if let Poll::Ready(Some(msg)) = this.engine_message_stream.poll_next_unpin(cx) {

we can change this to, draining the incoming messages and storing them in a vecque temporarily, and proceed to pop from the vecdque once we're ready to handle them

Additional context

No response

@mattsse mattsse added C-enhancement New feature or request C-perf A change motivated by improving speed, memory usage or disk footprint A-consensus Related to the consensus engine labels May 11, 2024
This was referenced May 11, 2024
@emhane emhane assigned rkrasiuk and unassigned emhane May 20, 2024
@emhane
Copy link
Member

emhane commented May 30, 2024

related #8251 (comment) @rkrasiuk

@emhane emhane reopened this May 31, 2024
@emhane emhane added C-bug An unexpected or incorrect behavior and removed C-perf A change motivated by improving speed, memory usage or disk footprint labels May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-consensus Related to the consensus engine C-bug An unexpected or incorrect behavior C-enhancement New feature or request
Projects
Status: In Progress
3 participants