New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a middleware to change the storageclass #625
Comments
This is a good feature request and something that is straightforward for some common use cases. S3Proxy uses Apache jclouds to enable portability between object stores. S3 has more storage tiers than jclouds as this mapping shows: public enum StorageClass {
STANDARD(Tier.STANDARD),
STANDARD_IA(Tier.INFREQUENT),
ONEZONE_IA(Tier.INFREQUENT),
INTELLIGENT_TIERING(Tier.STANDARD),
REDUCED_REDUNDANCY(Tier.STANDARD),
GLACIER(Tier.ARCHIVE),
GLACIER_IR(Tier.ARCHIVE),
DEEP_ARCHIVE(Tier.ARCHIVE);
} Azure support a Cold tier and GCS supports Coldline so adding one more level like But for the pure S3 use case, supporting all the storage classes would require adding non-portable logic to |
The former replaces INFREQUENT. References gaul/s3proxy#625.
The former replaces INFREQUENT. References gaul/s3proxy#625.
In CEPH you can create arbitrary name for the storageclass when you build your zone group placement even if it is not a good idea in my opinion. |
The former replaces INFREQUENT. References gaul/s3proxy#625.
Yeah I think this |
Some tools are configured to write to s3 with no option to configure the storageclass. This feature request will let those tools use the proxy with default or no storageclass and the proxy will write to the proxied s3 with a target storageclass.
We need that because some tools predates the creation of the storageclass feature brought up by AWS. We also have case where these tools has this option for AWS but not for S3 compatible, however minio or ceph let you build s3 solution with different storage class.
The natural approach would be to change those tools to add the storageclass information in their PUT request but often this feature is not prioritised by the development team for different reasons like upgrading their library create too many frictions with their existing codebase.
Here an example of how one would use this middleware
this configuration make sure that no information on the storageclass (default) or the information STANDARD storageclass is provided then s3proxy will write to the STANDARD_IA storageclass.
In this situation
Any request made for putting the object on GLACIER will be translated in HDD_SC.
This request should also work the other way, any GET request that return HDD_SC should be transformed in GLACIER .
The text was updated successfully, but these errors were encountered: