Support AWS SSO for S3 backend

[fdw]

Output of restic version

restic 0.16.2 compiled with go1.21.3 on linux/amd64

What should restic do differently? Which functionality do you think we should add?

Support AWS SSO logins for authenticating against the S3 backend

What are you trying to do? What problem would this solve?

I would like to use restic to backup data on an AWS S3 bucket that is protected using AWS SSO. AWS_PROFILE is already supported by restic, but the SSO login doesn't set the env variables directly. Instead, they're written to a json file in .aws/sso/cache. I would like restic to read and use them (or find another way to work frictionlessly with SSO).

Did restic help you today? Did it make you happy in any way?

Yes, restic does make me happy, so thank you for all your efforts! I feel a lot safer knowing (not just believing) that I have backups available.

[konidev20]

Hey @fdw,

If I got your requirement correctly.

aws sso login --profile myprofile creates the .aws/sso/cache file?

[Edit] Would be similar to az login and using Azure Blob storage [\Edit]

[fdw]

Yes, but it's a file in .aws/sso/cache with a random (as far as I can tell) name. Using the newest one should be fine.

[MichaelEischer]

We use https://github.com/minio/minio-go as library to access S3. If that has support for that SSO mechanism (not sure whether it does, I haven't checked), then I'm fine with adding support for AWS SSO.

[konidev20]

Hey found this has been discussed before on the restic forum: Is AWS SSO authentication supported

Quoting from the answer from Jul'22:

We using the minio go sdk and not the aws sdks. From a quick glance it looks like it would be necessary to introduce a completely new authentication flow to get credentials via SSO.

restic currently only supports the standard S3 access keys. If it is possible to retrieve such (temporary) keys for an SSO login, then it’s probably relatively simple to extend restic to support them. But someone would have to figure out the necessary steps and work on a pull request.

minio-go/pkg/credentials Looks like the status remains the same. Support has not been added. You can check the thread for some workarounds mentioned by another user.

[fdw]

That is quite a pity, as I have no alternative to SSO, and copying the keys every time I want to restic is not really a viable option. It only makes me want to avoid doing backups.

Thank you anyway 🙂

[konidev20]

I have created a feature request on the minio-go github,they can probably pick this up if they're interested or recommend any alternative way

Here is a reference to that issue:
Add ability to use AWS SSO profile as a credential provider

[konidev20]

aws-sso-file-credential-support.tar.gz

Hey @fdw,

[Edit] I have pulled from a pending MR on minio which implements this requirement; and have locally pulled the package.
[\Edit]

You can check the source code I've shared above. I was not sure how to share with your a working build, so I decided to just share the source code with you. You can run a local build.

1. Extract the tar

tar -xzvf aws-sso-file-credential-support.tar.gz 

2. Change directory into the restic folder

cd ./aws-sso-file-credential-support/restic

3. Run go mod tidy

go mod tidy

4. Run go build script

go run build.go

PS: @MichaelEischer please let me know if this is a nice way of sharing some test code, I couldn't think of anything else.

[fdw]

I just tried it, but unfortunately I only get errors: With init, it's create repository at s3:s3.amazonaws.com/<bucket>/<path> failed: client.BucketExists: 301 Moved Permanently.
With backup, I get:

create repository at s3:s3.amazonaws.com/<bucket>/<path>/ failed: client.BucketExists: 301 Moved Permanently

This sounds like a rights issue :/ The aws s3 cli has no problem accessing the bucket.

[MichaelEischer]

@fdw Please use the region specific S3 endpoint, e.g. s3.us-west-2.amazonaws.com/bucket_name

[fdw]

Unfortunately, s3:s3.eu-central-1.amazonaws.com/<bucket>/<path> has the same result.

[konidev20]

@fdw I guess, you can mention this issue on the minio thread. Probably the original creator of the sso method might help with something.

[jenic]

@fdw I use restic with aws-vault and sso (aws-vault exec backup -- restic backup ...), maybe that will at least allow you to use restic easily until it's officially supported?