Document how to use the Amazon S3 Media module with a local storage emulator, implement support for it if necessary

[Piedone]

Is your feature request related to a problem? Please describe.

Following the docs, you can only use the Amazon S3 Media module with a real, online S3 resource. For daily development, especially in a team, using a storage emulator is better though. This is currently an issue only for the Amazon Media Storage feature, but after #15028, it'll be for Amazon Media ImageSharp Image Cache too.

AFAIK, with the AWS SDK version we use currently, and due to us using virtual host addressing for buckets (i.e. mybucket.localhost, as opposed to path-style addressing with localhost/mybucket) this isn't actually possible, so we need to implement support for it too. I elaborate my attempts below.

• S3rver can be made work as far as creating and accessing buckets with the below config, but file uploads fail with “Expected hash not equal to calculated hash”. This is related to us using the almost latest AWS SDK and a breaking change in it; unfortunately, due to S3rver being unmaintained, it's hopeless that it'll be fixed.

appsettings:

    "OrchardCore_Media_AmazonS3": {
      "ServiceURL": "http://localhost:4568/",
      "Profile": "default",
      "ProfilesLocation": "",
      "Credentials": {
        "SecretKey": "S3RVER",
        "AccessKey": "S3RVER"
      },
      "BasePath": "/{{ ShellSettings.Name }}",
      "CreateBucket": true,
      "RemoveBucket": true, // Whether the 'Bucket' is deleted if the tenant is removed, false by default.
      "BucketName": "ocmedia"
    },
    // See https://docs.orchardcore.net/en/latest/docs/reference/modules/Media.AmazonS3/#configuration to configure media storage in Amazon S3 Storage.
    "OrchardCore_Media_AmazonS3_ImageSharp_Cache": {
      "ServiceURL": "http://localhost:4568/",
      "Profile": "default",
      "ProfilesLocation": "",
      "Credentials": {
        "SecretKey": "S3RVER",
        "AccessKey": "S3RVER"
      },
      "BasePath": "/{{ ShellSettings.Name }}",
      "CreateBucket": true,
      "RemoveBucket": true, // Whether the 'Bucket' is deleted if the tenant is removed, false by default.
      "BucketName": "ocimagesharp"
    },

Windows hosts:

::1 ocmedia.localhost
::1 ocimagesharp.localhost

• S3 ninja with docker run -p 9444:9000 scireum/s3-ninja:latest: Doesn't seem to support virtual host addressing, see Virtual host address support scireum/s3ninja#244. Using the below Windows hosts file doesn't help either. Furthermore, file uploads fail with “Expected hash not equal to calculated hash”. This is a known issue, see All Rejected after updating AwsSdk.S3 library scireum/s3ninja#240.

127.0.0.1 ocmedia.localhost
127.0.0.1 ocimagesharp.localhost

• LocalS3 doesn't have the “Expected hash not equal to calculated hash” issue but it doesn't support virtual host addressing either: Virtual host address support Robothy/local-s3#92.
• S3Mock doesn’t support host addressing either: https://github.com/adobe/S3Mock?tab=readme-ov-file#path-style-vs-domain-style-access.

Describe the solution you'd like

Document how a storage emulator can be used. However, as detailed above, most possibly none of the existing emulators will work.

Describe alternatives you've considered

Alternatively, we’d need to use path-style buckets with ForcePathStyle, see aws/aws-sdk-net#2571, but that we can’t really set in the way we currently use the AWS SDK (at least I didn't find a way to do that without completely overhauling the code). See Configure S3 client ServiceURL with AWSCredentials or RegionEndPoint.

[Piedone]

I couldn't leave this be and I think I managed to solve it. Will open a PR.