Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure path style in S3 binding #7394

Open
RaulGracia opened this issue Feb 15, 2024 · 0 comments
Open

Configure path style in S3 binding #7394

RaulGracia opened this issue Feb 15, 2024 · 0 comments
Assignees

Comments

@RaulGracia
Copy link
Contributor

Describe the bug
In some deployments of storage systems implementing the S3 API, the default "virtual path" style for request URLs does not work, as it may induce problems related to DNS. In these cases, the "path-style" URLs may be working fine, so it would be interesting to be able to configure the style of URLs for S3 biding as a parameter.

To Reproduce
Using "virtual-path" on a cluster with a storage system implementing S3 APIs, I got the following issue:

Caused by: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
...
	... 5 common frames omitted
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: [BUCKET_NAME].[STORAGE_SERVER_NAME]
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)

After modifying the S3 binding configuration to use "path-style" requests, Pravega was able to run correctly.

Screenshots
n/a

Additional information
A solution would be just adding something like this in the S3 binding configuration:

s3Client = S3Client.builder()
            .region(awsRegionProvider.region)
            .credentialsProvider(awsCredentialsProvider)
            .serviceConfiguration(
                S3Configuration.builder()
                .pathStyleAccessEnabled(enablePathStyleAccess)
                .build()
            ).build()

Where enablePathStyleAccess is a configurable parameter (deafulted to false as it is today).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant