Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QUIC support #557

Open
jpds opened this issue Mar 30, 2023 · 3 comments · May be fixed by #560
Open

QUIC support #557

jpds opened this issue Mar 30, 2023 · 3 comments · May be fixed by #560

Comments

@jpds
Copy link
Contributor

jpds commented Mar 30, 2023

I use s5cmd against my own S3-compliant storage which also has QUIC / HTTP/3 support on the frontend traefik servers.

Whilst Amazon S3 doesn't support this, it would be neat if s5cmd could integrate with https://github.com/quic-go/quic-go and upgrade connections to QUIC if support is detected.

@jpds
Copy link
Contributor Author

jpds commented Mar 30, 2023

I've tried implementing this myself, and it appears to be working with just:

diff --git a/storage/s3.go b/storage/s3.go
index 701b525..310d94c 100644
--- a/storage/s3.go
+++ b/storage/s3.go
@@ -29,6 +29,8 @@ import (
        "github.com/aws/aws-sdk-go/service/s3/s3iface"
        "github.com/aws/aws-sdk-go/service/s3/s3manager"
        "github.com/aws/aws-sdk-go/service/s3/s3manager/s3manageriface"
+       "github.com/quic-go/quic-go"
+       "github.com/quic-go/quic-go/http3"

        "github.com/peak/s5cmd/log"
        "github.com/peak/s5cmd/storage/url"
@@ -881,6 +883,16 @@ func (sc *SessionCache) newSession(ctx context.Context, opts Options) (*session.
        }

        var httpClient *http.Client
+
+       var qconf quic.Config
+       roundTripper := &http3.RoundTripper{
+               QuicConfig: &qconf,
+       }
+
+       httpClient = &http.Client{
+               Transport: roundTripper,
+       }
+
        if opts.NoVerifySSL {
                httpClient = insecureHTTPClient
        }

However it does make the client complain about buffer sizes per this upstream document.

I shall experiment with this a bit more.

jpds added a commit to jpds/s5cmd that referenced this issue Apr 24, 2023
@jpds jpds linked a pull request Apr 24, 2023 that will close this issue
jpds added a commit to jpds/s5cmd that referenced this issue Apr 24, 2023
@pr0ton11
Copy link

pr0ton11 commented May 2, 2023

Very good work, please keep us updated once it's implemented. What kind of S3 compliant storage do you use if I may ask?

jpds added a commit to jpds/s5cmd that referenced this issue Jul 3, 2023
jpds added a commit to jpds/s5cmd that referenced this issue Jul 3, 2023
jpds added a commit to jpds/s5cmd that referenced this issue Jul 3, 2023
@jpds
Copy link
Contributor Author

jpds commented Jul 3, 2023

@pr0ton11 Garage.

jpds added a commit to jpds/s5cmd that referenced this issue Jul 3, 2023
jpds added a commit to jpds/s5cmd that referenced this issue Jul 28, 2023
jpds added a commit to jpds/s5cmd that referenced this issue Jul 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants