Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support Workload Identity Federation on AWS ECS/Fargate #1374

Closed
wants to merge 1 commit into from
Closed

feat: Support Workload Identity Federation on AWS ECS/Fargate #1374

wants to merge 1 commit into from

Conversation

jurgispods
Copy link

@jurgispods jurgispods commented Mar 7, 2024 •
edited

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #957 ☕️

This PR adds functionality to detect if the code is running in an ECS container. In that case, the private container metadata endpoint is used to obtain temporary credentials.

This makes it possible to use Workload Identity Federation on ECS without using a workaround like manually obtaining temporary session credentials and exporting them as env vars (see here).

I was able to successfully test this PR by running an ECS task which connects to Bigquery via Workload Identity Federation.

@jurgispods jurgispods requested review from a team as code owners March 7, 2024 14:46
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Mar 7, 2024
@jurgispods jurgispods mentioned this pull request Mar 7, 2024
Open
@lsirac
Copy link
Collaborator

lsirac commented Mar 12, 2024 •
edited

Hi @jurgispods, we support custom credential suppliers. This is the approach we recommend for now.

@jurgispods
Copy link
Author

jurgispods commented Mar 18, 2024 •
edited

Hi @lsirac, thanks for the heads-up. I guess I could provide a custom credential supplier instead, but that would require the users to manually instantiate the AwsCredentials using the custom supplier.

It seems quite a lot of people are interested in using Workload Identity Federation from ECS, so do you think there would be a case for this library to support ECS directly instead of the user having to provide a custom supplier? In many cases, it is very hard to add custom code to an application, since the GCS auth does not happen inside your own codebase. For example, in my case I am running a Kafka Bigquery connector, so the changes would have to happen in the connector's codebase.

From a user point of view, it would be most convenient if we could configure everything from outside (i.e. just GOOGLE_APPLICATION_CREDENTIALS) instead of providing a custom supplier per AWS runtime. I assume that you are reluctant to add specifics of obtaining credentials for each and every AWS service that exists? What about supporting at least the most popular ones, like ECS and Lambda?

@lsirac
Copy link
Collaborator

lsirac commented Apr 9, 2024

Yes, we may add it in the future but it would be part of a larger overhaul. I am going to close this for now. If we do add support for this I will tag you.

@lsirac lsirac closed this Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TimurSadykov TimurSadykov Awaiting requested review from TimurSadykov

@westarle westarle Awaiting requested review from westarle

@lsirac lsirac Awaiting requested review from lsirac

Assignees

@lsirac lsirac

Labels
size: m Pull request size is medium.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Workload Identity Federation should support ECS Fargate container credentials
2 participants
@jurgispods @lsirac