-
Notifications
You must be signed in to change notification settings - Fork 796
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Java-based S3 multipart client as an altertivate to AWS CRT-based S3 client #5177
Comments
Hi @aymkhalil, this is by design. The AWS CRT-based S3 client is designed to maximize performance and the CRT component it depends on is written in C, and adding execution interceptor may downgrade performance since it'd introduce more JNI hops. We are currently working on a Java-based multipart client utilizing multipart upload/download to achieve parallel transfers. It may not offer the same throughput as the CRT S3 client, but it supports all configuration settings that are available in the vanilla S3 client such as execution interceptor. |
@zoewangg thanks for the clarification. The Java-based multipart client is something we'd be very interested in, if you have more info on where/how to track that initiative, that would be great. |
We can use this issue to track since I don't think there's an existing issue. I'll update the title. It currently only supports upload and we are actively working on download.
|
ExecutionInterceptor
support to the CRT-based S3 Client Builder S3AsyncClient.crtBuilder()
Hi team, we use an execution interceptor with our client to propagate tracing through the framework. Can this be supported in the CRT client without affecting performance? |
Describe the feature
The standard
S3AsyncClient
builder allows for overriding sdk configs which allows for adding customExecutionInterceptors
. The CRT builder is lacking the capability to inject interceptors. This hinders upgrade efforts from the standard client to the CRT based client.Use Case
We use it with the standard client mainly for injecting test failures.
ExecutionInterceptor
are very handy to test how our systems reacts under simulated failure conditions.Proposed Solution
It would be ideal if
overrideClientConfig
is exposed so the CRT clients is one step closer to being a drop in replacement for the standard client. But I've seen other configs added a la carte as top level builder configs (e.g. theretryPolicy
) so maybeExecutionInterceptor
fits there as well.Other Information
No response
Acknowledgements
AWS Java SDK version used
2.25.40
JDK version used
java version "17.0.3.1" 2022-04-22 LTS
Operating System and version
M1 Mac
The text was updated successfully, but these errors were encountered: