Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds the limit for the deferred stream's queue size #6852

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

vdeweb
Copy link

@vdeweb vdeweb commented Jan 24, 2024

Fixes the issue with streaming: if the rate of consuming data by the client is slower than the rate of streaming data by the server, the server prefetches too much data from the source.

Proposed solution is to limit the queue size (size of the result buffer) in DeferredWorkState.

Changes:

  • size of the queue in DeferredWorkState has a limit now;
  • action of registering DeferredExecutionTask is unbound from its starting with new TaskDispatcher class;
  • previous DeferredExecutionTask has to complete before starting the new one in DeferredStream.

@CLAassistant
Copy link

CLAassistant commented Jan 24, 2024

CLA assistant check
All committers have signed the CLA.

Copy link

codecov bot commented Jan 24, 2024

Codecov Report

Attention: 25 lines in your changes are missing coverage. Please review.

Comparison is base (acffe7b) 73.15% compared to head (0decfd5) 73.13%.
Report is 4 commits behind head on main.

Files Patch % Lines
...te/Core/src/Execution/Processing/DeferredStream.cs 0.00% 8 Missing ⚠️
...escriptors/Conventions/XmlDocumentationProvider.cs 0.00% 8 Missing ⚠️
...Core/src/Execution/Processing/DeferredWorkState.cs 81.25% 3 Missing ⚠️
...cyInjection/InternalServiceCollectionExtensions.cs 75.00% 2 Missing ⚠️
.../src/Execution/Processing/DeferredWorkScheduler.cs 66.66% 2 Missing ⚠️
...Execution/Processing/Tasks/ResolverTask.Execute.cs 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6852      +/-   ##
==========================================
- Coverage   73.15%   73.13%   -0.02%     
==========================================
  Files        2499     2499              
  Lines      126162   126170       +8     
==========================================
- Hits        92292    92273      -19     
- Misses      33870    33897      +27     
Flag Coverage Δ
unittests 73.13% <55.35%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@michaelstaib
Copy link
Member

We still have this on the back log. I will soon start work on the defer / stream part and then we will have a look at this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants