Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HTTP/3] Support for HTTP/3 multiple connections #101531

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

ManickaP
Copy link
Member

@ManickaP ManickaP commented Apr 25, 2024

Feel free to review and discuss the API changes. This is NO MERGE until we have the APIs approved and all the TODOs resolved

The implementation takes H/2 connection pooling and slightly adjusts it for H/3 specific behaviors (see a diff between HttpConnectionPool.Http2.cs and HttpConnectionPool.Http3.cs).

The PR depends on 2 API reviews:

TODO: Add numbers from benchmarks when it runs
TODO: Stress run depends on dotnet/aspnetcore#55282

Fixes #51775
Fixes #54968
Fixes #68380
Resolves #101535
Resolves #101534

Copy link

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ManickaP ManickaP added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Apr 25, 2024
@ManickaP ManickaP requested a review from a team April 25, 2024 08:30
@ManickaP
Copy link
Member Author

ManickaP commented Apr 25, 2024

No noticeable regression on perf from main:

main

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 35                                    |
| Max Cores usage (%)         | 971                                   |
| Max Working Set (MB)        | 3,810                                 |
| Max Private Memory (MB)     | 4,415                                 |
| Build Time (ms)             | 6,988                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24224.9             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 519                                   |
| Requests                    | 345,446                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 23,069                                |
  1. 1x100 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 41                                    |
| Max Cores usage (%)         | 1,150                                 |
| Max Working Set (MB)        | 577                                   |
| Max Private Memory (MB)     | 1,119                                 |
| Build Time (ms)             | 4,926                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 520                                   |
| Requests                    | 989,468                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,955                                |
  1. 20x500 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 97                                    |
| Max Cores usage (%)         | 2,710                                 |
| Max Working Set (MB)        | 6,526                                 |
| Max Private Memory (MB)     | 7,268                                 |
| Build Time (ms)             | 2,544                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 518                                   |
| Requests                    | 4,976,883                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 331,779                               |

PR, multiple H3 connections OFF

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 36                                    |
| Max Cores usage (%)         | 998                                   |
| Max Working Set (MB)        | 3,772                                 |
| Max Private Memory (MB)     | 4,371                                 |
| Build Time (ms)             | 2,866                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 596                                   |
| Requests                    | 342,841                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 22,914                                |
  1. 1x100 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=100 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dl
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 46                                    |
| Max Cores usage (%)         | 1,296                                 |
| Max Working Set (MB)        | 587                                   |
| Max Private Memory (MB)     | 1,174                                 |
| Build Time (ms)             | 2,897                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 985,119                               |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 65,691                                |
  1. 20x500 (clients x threads)
rank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=20 --variable concurrencyPerHttpClient=500 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                       |
| --------------------------- | ------------------------------------- |
| Max CPU Usage (%)           | 95                                    |
| Max Cores usage (%)         | 2,667                                 |
| Max Working Set (MB)        | 6,494                                 |
| Max Private Memory (MB)     | 7,246                                 |
| Build Time (ms)             | 2,660                                 |
| Start Time (ms)             | 0                                     |
| Published Size (KB)         | 74,115                                |
| Symbols Size (KB)           | 19                                    |
| .NET Core SDK Version       | 9.0.100-preview.5.24225.1             |
| ASP.NET Core Version        | 9.0.0-preview.4.24223.1+f18510c2fbdf  |
| .NET Runtime Version        | 9.0.0-preview.4.24223.11+d92ac1f892a7 |
| Processor Count             | 28                                    |
| First request duration (ms) | 594                                   |
| Requests                    | 5,334,724                             |
| Bad Status Code Requests    | 0                                     |
| Exceptions                  | 0                                     |
| Mean RPS                    | 355,707                               |

PR, multiple H3 connections ON

  1. 1x10000 (clients x threads)
crank --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/ci.profile.yml --config https://raw.githubusercontent.com/aspnet/Benchmarks/main/scenarios/httpclient.benchmarks.yml --client.framework net9.0 --server.framework net9.0 --scenario httpclient-kestrel-get --profile intel-lin-app --profile amd-lin2-load --variable useHttpMessageInvoker=true --variable httpVersion=3.0 --variable useHttps=true --variable responseSize=256 --variable numberOfHttpClients=1 --variable concurrencyPerHttpClient=10000 --client.options.outputFiles artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Http.dll --client.options.outputFiles  artifacts/bin/testhost/net9.0-linux-Release-x64/shared/Microsoft.NETCore.App/9.0.0/System.Net.Quic.dll
| client                      |                                      |
| --------------------------- | ------------------------------------ |
| Max CPU Usage (%)           | 92                                   |
| Max Cores usage (%)         | 2,572                                |
| Max Working Set (MB)        | 6,491                                |
| Max Private Memory (MB)     | 7,228                                |
| Build Time (ms)             | 2,736                                |
| Start Time (ms)             | 0                                    |
| Published Size (KB)         | 74,777                               |
| Symbols Size (KB)           | 19                                   |
| .NET Core SDK Version       | 9.0.100-preview.5.24272.19           |
| ASP.NET Core Version        | 9.0.0-preview.6.24272.7+49c1c68bf1ac |
| .NET Runtime Version        | 9.0.0-preview.5.24272.2+5c06e5d01fa0 |
| Processor Count             | 28                                   |
| First request duration (ms) | 594                                  |
| Requests                    | 4,325,325                            |
| Bad Status Code Requests    | 0                                    |
| Exceptions                  | 0                                    |
| Mean RPS                    | 288,664                              |

@ManickaP
Copy link
Member Author

Change of the logic: StreamsAvailable has been made a regular callback and moved to QuicConnectionOptions (thanks @rzikm for the suggestion). This is more aligned with how we handle callbacks/events in SslStream. After all, events are not a common occurrence in our stack. It also solves an issue with the correctness of the provided values in regards to thread safety. There's no chance of registering to the event in the middle of opening stream or processing STREAMS_AVAILABLE event. The callback is registered before the connection is created and the callback is always invoked as a reaction to a MsQuic event. The only thing that could happen is that invocations of the callback might swap order since we must offload the callback execution from MsQuic thread to the thread-pool. However, we are providing increments, so it doesn't matter that much as a+b == b+a.

ManickaP added a commit to dotnet/dotnet-api-docs that referenced this pull request May 16, 2024
ManickaP added a commit to dotnet/dotnet-api-docs that referenced this pull request May 16, 2024
Copy link
Member

@MihaZupan MihaZupan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few more comments but otherwise this LGTM.

It's good that it substantially mimics the H2 logic, makes it much easier to follow.

@@ -210,6 +284,7 @@ public async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, lon

if (quicStream == null)
{
ReleaseStream();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a 100% reliable way to detect whether the underlying QuicConnection counted the stream as consumed or not?
E.g. even when OpenOutboundStreamAsync is cancelled?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, this is not actually right, in both cases we should not re-increment the count (only if we didn't even try to open the stream). I did some testing and added some comments to QuicStream to make sure we have consistent behavior wrt stream counts regardless whether the call gets cancelled or not.

}
public delegate void QuicConnectionStreamsAvailableCallback(System.Net.Quic.QuicConnection connection, int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More of an API review question, but is there any chance we'll want to expose more info here such that we'd want the arguments to be a struct instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have only stream count related info, technically the transport sends MAX_STREAM_ID and separately for Uni/Bi directional streams.
The only think we could expose are different ways to express the new stream limit, which is part of alternative design in #101534

/// Provided via <see cref="StartAsync(Action{QuicStreamType}, CancellationToken)" /> from <see cref="QuicConnection" /> so that <see cref="QuicStream"/> can decrement its available stream count field.
/// When <see cref="HandleEventStartComplete(ref START_COMPLETE_DATA)">START_COMPLETE</see> arrives it gets invoked and unset back to <c>null</c> to not to hold any unintended reference to <see cref="QuicConnection"/>.
/// </summary>
private Action<QuicStreamType>? _decrementAvailableStreamCount;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this functionally the same as if we instead temporarily stored a QuicConnection and exposed the decrement function there as internal?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it is. The intent here was to express that we need the link only until the stream opening (=reserving the Stream ID) is confirmed by MsQuic. After that, the delegate gets nulled, destroying the link. But let's wait for the API review of #101534 and if we stick with the increments, I'll try to push this to the MsQuic layer as you suggested and this will moot.

@@ -603,6 +674,15 @@ private unsafe int HandleEventPeerStreamStarted(ref PEER_STREAM_STARTED_DATA dat
data.Flags |= QUIC_STREAM_OPEN_FLAGS.DELAY_ID_FC_UPDATES;
return QUIC_STATUS_SUCCESS;
}
private unsafe int HandleEventStreamsAvailable(ref STREAMS_AVAILABLE_DATA data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get MsQuic to expose increments instead of current counts?
(presumably, that's the data that they're reacting to that triggered these events in the first place?)

That'd let us drop the whole DecrementAvailableStreamCount dance

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. I can try that. It would also resolve the issue with the _decrementAvailableStreamCount callback. However, I'll not push on adding this to MsQuic, until we have the API approved, we might end up with MAX_STREAM_ID or different value being reported by the callback...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft change in 3621064
MsQuic change in microsoft/msquic@d47f9c1

@@ -427,7 +498,7 @@ public async ValueTask<QuicStream> OpenOutboundStreamAsync(QuicStreamType type,
NetEventSource.Info(this, $"{this} New outbound {type} stream {stream}.");
}

await stream.StartAsync(cancellationToken).ConfigureAwait(false);
await stream.StartAsync(DecrementAvailableStreamCount, cancellationToken).ConfigureAwait(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using DecrementAvailableStreamCount directly here will allocate a new object for every stream.
We could cache a single Action<QuicStreamType> instance on the connection.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this open as this is all tied together with the other comments related to this delegate.

connectionException = e is OperationCanceledException oce && oce.CancellationToken == cts.Token && !waiter.CancelledByOriginatingRequestCompletion ?
CreateConnectTimeoutException(oce) :
e;
// If the connection hasn't been initialized with QuicConnection there's no need to dispose it.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes that InitQuicConnection may never throw, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it got me thinking and it looks like, we don't need this in the end as the H/3Connection Dispose does check for _connection not being null, so calling Dispose on not fully initialized connection should not do any harm.

@ManickaP
Copy link
Member Author

/azp list

Copy link

CI/CD Pipelines for this repository:

@ManickaP
Copy link
Member Author

/azp run runtime-libraries-coreclr outerloop

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ManickaP
Copy link
Member Author

/azp run runtime-libraries stress-http

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ManickaP
Copy link
Member Author

/azp run runtime

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-System.Net.Http new-api-needs-documentation NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons)
Projects
None yet
3 participants