Skip to content

AWS S3 Storage

Robin Rodricks edited this page May 14, 2024 · 24 revisions

In order to use AWS S3 or S3-compatible storage you need to reference NuGet package first. The provider wraps around the standard AWS SDK which is updated regularly, but adds a lot of untrivial workarounds that makes your life painless.

Connect to AWS S3

You can use the AwsS3 methods to connect to AWS S3 storage.

IBlobStorage storage = StorageFactory.Blobs.AwsS3(
    accessKeyId, secretAccessKey, sessionToken, bucketName);

Connect to DigitalOcean Spaces

You can use the DigitalOceanSpaces method to connect to DigitalOcean Spaces storage (S3 compatible).

For digitalOceanRegion, you need to enter the DigitalOcean Region endpoint, like nyc3. This is internally combined into the following string which is used as the Service URL: "https://{digitalOceanRegion}.digitaloceanspaces.com".

IBlobStorage storage = StorageFactory.Blobs.DigitalOceanSpaces(
    accessKeyId, secretAccessKey, bucketName, digitalOceanRegion);

Connect to MinIO

You can use the MinIO method to connect to MinIO storage servers (S3 compatible).

For minioServerUrl, you need to enter the absolute server URL of your MinIO endpoint.

IBlobStorage storage = StorageFactory.Blobs.MinIO(
    accessKeyId, secretAccessKey, bucketName, awsRegion, minioServerUrl);

Connect to Wasabi

You can use the Wasabi method to connect to Wasabi storage (S3 compatible).

To obtain the wasabiServiceUrl, refer to this Wasabi article, and take the values in the Service URL column (add https:// as a prefix).

IBlobStorage storage = StorageFactory.Blobs.Wasabi(
    accessKeyId, secretAccessKey, bucketName, wasabiServiceUrl);

Connection Strings

To create with a connection string, first reference the module:

StorageFactory.Modules.UseAwsStorage();

Then construct using the following format:

IBlobStorage storage = StorageFactory.Blobs.FromConnectionString("aws.s3://keyId=...;key=...;bucket=...;region=...");

where:

  • keyId is (optional) access key ID.
  • key is (optional) secret access key.
  • bucket is bucket name.
  • region is the AWS account region (such as us-east-1).
  • serviceUrl is the optional URL of the storage provider (for example for DigitalOcean).

If keyId and key are omitted, the AWS SDK's default approach to credential resolution will be used. For example: if running in Lambda, it will assume the Lambda execution role; if there are credentials configured in ~/.aws, it will use those; etc. See AWS SDK Documentation for more details.

AWS CLI Profile Support

If you already have credentials in the local ~/.aws/credentials generated by AWS CLI, you can also use them to connect to a bucket with FluentStorage. Credentials file example:

[default]
aws_access_key_id = ASIAV44FYRYLSBYIPG67
aws_secret_access_key = zhuh8nSUAR0pR5jbi4vhUym95JsC7ay3iiE2v9Jq
aws_session_token = FQoGZXIvY...

[anotherProfile]
aws_access_key_id = ASIAXEKXYP2H7MRN45FD
aws_secret_access_key = 8i6jnOOurRtTiK4CWN+uUD115SYS8++ZuQjO/LtW
aws_session_token = FQoGZXI...

...

To connect using say anotherProfile credentials, construct the instance by:

IBlobStorage bs = StorageFactory.Blobs.AwsS3(
   "anotherProfile",
   "bucketName",
   "region");

AwsCliCredentials class contains credentials file parrsing logic and some utility methods you mind find useful, for instance enumerating available profiles and so on.

Native Operations

Native operations are exposed via IAwsS3BlobStorageNativeOperations interface.