Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add workload identity federation support #547

Merged
merged 30 commits into from Feb 18, 2021
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e3b2751
feat: add STS token exchange utility (#454)
lsirac Aug 5, 2020
ebb4486
feat: adds support for 3PI credentials (#464)
lsirac Aug 26, 2020
4ee1a14
feat: implements AWS signature version 4 for signing requests (#476)
lsirac Sep 30, 2020
d639a78
Merge branch 'master' into 3pi
chingor13 Oct 12, 2020
4666949
feat: support generic token formats in IdentityPoolCredentials (#484)
lsirac Oct 15, 2020
9d4d721
feat: adds support for AWS credentials (#483)
lsirac Oct 15, 2020
f7f741a
Merge branch 'master' into 3pi
chingor13 Oct 15, 2020
dc16550
feat: add quota project ID to requestMetadata if present (#495)
lsirac Oct 15, 2020
6d04b05
chore: remove use of Truth assertions (#498)
lsirac Oct 16, 2020
124c77c
feat: add external account credentials to ADC (#500)
lsirac Oct 21, 2020
17e849e
chore: use ImpersonatedCredentials for service account impersonation …
lsirac Oct 29, 2020
223c6ec
fix: fix issues found through manual testing (#506)
lsirac Nov 17, 2020
478f33b
Merge branch 'master' into 3pi
chingor13 Dec 14, 2020
3769b90
fix: updates AWS credential source (#520)
lsirac Dec 15, 2020
0ccd43d
Merge branch 'master' into 3pi
chingor13 Jan 28, 2021
48405f7
fix: merge
lsirac Jan 28, 2021
e481da2
fix: change copyright year to 2021
lsirac Jan 29, 2021
36ff0f6
fix: address review comments
lsirac Jan 29, 2021
b9b9f8d
fix: remove assertThrows
lsirac Feb 2, 2021
a980a19
fix: address review comments
lsirac Feb 3, 2021
6c3982c
fix: review comments
lsirac Feb 12, 2021
25c884b
fix: more review comments
lsirac Feb 12, 2021
bd49373
fix: review
lsirac Feb 13, 2021
de0960d
fix: review
lsirac Feb 13, 2021
4d384b9
Merge branch 'master' of https://github.com/googleapis/google-auth-li…
lsirac Feb 17, 2021
c9ee282
fix: review
lsirac Feb 17, 2021
623d878
fix: add CredentialFormatException
lsirac Feb 17, 2021
1fe5188
fix: review
lsirac Feb 17, 2021
56a2e2b
Merge branch 'master' of https://github.com/googleapis/google-auth-li…
lsirac Feb 18, 2021
c7f5730
Merge branch 'master' into byoid
chingor13 Feb 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
348 changes: 348 additions & 0 deletions oauth2_http/java/com/google/auth/oauth2/AwsCredentials.java
@@ -0,0 +1,348 @@
/*
* Copyright 2021 Google LLC
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
*
* * Neither the name of Google LLC nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

package com.google.auth.oauth2;

import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpResponse;
import com.google.api.client.json.GenericJson;
import com.google.api.client.json.JsonParser;
import com.google.auth.http.HttpTransportFactory;
import com.google.common.annotations.VisibleForTesting;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.annotation.Nullable;

/**
* AWS credentials representing a third-party identity for calling Google APIs.
*
* <p>By default, attempts to exchange the external credential for a GCP access token.
*/
public class AwsCredentials extends ExternalAccountCredentials {
lsirac marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this class need to be public? It looks like the only public methods are overridden or visible for testing, so I'm guessing no. The less API we have to publish and support ever after the better.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to keep this public. Developers can initialize an AwsCredential and use it directly, and other credentials follow that pattern. They also need to be able to create an AwsCredential with specific scopes.


/**
* The AWS credential source. Stores data required to retrieve the AWS credential from the AWS
* metadata server.
*/
static class AwsCredentialSource extends CredentialSource {

private final String regionUrl;
private final String url;
private final String regionalCredentialVerificationUrl;

/**
* The source of the AWS credential. The credential source map must contain the
* `regional_cred_verification_url` field.
*
* <p>The `regional_cred_verification_url` is the regional GetCallerIdentity action URL, used to
* determine the account ID and its roles.
*
* <p>The `environment_id` is the environment identifier, in the format “aws${version}”. This
* indicates whether breaking changes were introduced to the underlying AWS implementation.
*
* <p>The `region_url` identifies the targeted region. Optional.
*
* <p>The `url` locates the metadata server used to retrieve the AWS credentials. Optional.
*/
AwsCredentialSource(Map<String, Object> credentialSourceMap) {
super(credentialSourceMap);
if (!credentialSourceMap.containsKey("regional_cred_verification_url")) {
throw new IllegalArgumentException(
"A regional_cred_verification_url representing the"
+ " GetCallerIdentity action URL must be specified.");
}

String environmentId = (String) credentialSourceMap.get("environment_id");

// Environment version is prefixed by "aws". e.g. "aws1".
Matcher matcher = Pattern.compile("^(aws)([\\d]+)$").matcher(environmentId);
if (!matcher.find()) {
lsirac marked this conversation as resolved.
Show resolved Hide resolved
throw new IllegalArgumentException("Invalid AWS environment ID.");
}

int environmentVersion = Integer.parseInt(matcher.group(2));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more simply, this looks like the string simply equals the literal "aws1"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm we have to parse the version anyway for the error message

if (environmentVersion != 1) {
throw new IllegalArgumentException(
String.format(
"AWS version %s is not supported in the current build.", environmentVersion));
}

this.regionUrl = (String) credentialSourceMap.get("region_url");
this.url = (String) credentialSourceMap.get("url");
this.regionalCredentialVerificationUrl =
(String) credentialSourceMap.get("regional_cred_verification_url");
}
}

/**
* Internal constructor. See {@link
* ExternalAccountCredentials#ExternalAccountCredentials(HttpTransportFactory, String, String,
* String, CredentialSource, String, String, String, String, String, Collection)}
*/
AwsCredentials(
HttpTransportFactory transportFactory,
String audience,
String subjectTokenType,
String tokenUrl,
AwsCredentialSource credentialSource,
@Nullable String tokenInfoUrl,
@Nullable String serviceAccountImpersonationUrl,
@Nullable String quotaProjectId,
@Nullable String clientId,
@Nullable String clientSecret,
@Nullable Collection<String> scopes) {
super(
transportFactory,
audience,
subjectTokenType,
tokenUrl,
credentialSource,
tokenInfoUrl,
serviceAccountImpersonationUrl,
quotaProjectId,
clientId,
clientSecret,
scopes);
}

@Override
public AccessToken refreshAccessToken() throws IOException {
StsTokenExchangeRequest.Builder stsTokenExchangeRequest =
StsTokenExchangeRequest.newBuilder(retrieveSubjectToken(), subjectTokenType)
.setAudience(audience);

// Add scopes, if possible.
if (scopes != null && !scopes.isEmpty()) {
stsTokenExchangeRequest.setScopes(new ArrayList<>(scopes));
}

return exchangeExternalCredentialForAccessToken(stsTokenExchangeRequest.build());
}

@Override
public String retrieveSubjectToken() throws IOException {
// The targeted region is required to generate the signed request. The regional
// endpoint must also be used.
String region = getAwsRegion();

AwsSecurityCredentials credentials = getAwsSecurityCredentials();

// Generate the signed request to the AWS STS GetCallerIdentity API.
Map<String, String> headers = new HashMap<>();
headers.put("x-goog-cloud-target-resource", audience);

AwsRequestSigner signer =
AwsRequestSigner.newBuilder(
credentials,
"POST",
((AwsCredentialSource) credentialSource)
.regionalCredentialVerificationUrl.replace("{region}", region),
region)
.setAdditionalHeaders(headers)
.build();

AwsRequestSignature awsRequestSignature = signer.sign();
return buildSubjectToken(awsRequestSignature);
}

/** Clones the AwsCredentials with the specified scopes. */
@Override
public GoogleCredentials createScoped(Collection<String> newScopes) {
return new AwsCredentials(
transportFactory,
audience,
subjectTokenType,
tokenUrl,
(AwsCredentialSource) credentialSource,
tokenInfoUrl,
serviceAccountImpersonationUrl,
quotaProjectId,
clientId,
clientSecret,
newScopes);
}

private String retrieveResource(String url, String resourceName) throws IOException {
try {
HttpRequestFactory requestFactory = transportFactory.create().createRequestFactory();
HttpRequest request = requestFactory.buildGetRequest(new GenericUrl(url));
HttpResponse response = request.execute();
return response.parseAsString();
} catch (IOException e) {
throw new IOException(String.format("Failed to retrieve AWS %s.", resourceName), e);
}
}

private String buildSubjectToken(AwsRequestSignature signature)
throws UnsupportedEncodingException {
Map<String, String> canonicalHeaders = signature.getCanonicalHeaders();
List<GenericJson> headerList = new ArrayList<>();
for (String headerName : canonicalHeaders.keySet()) {
headerList.add(formatTokenHeaderForSts(headerName, canonicalHeaders.get(headerName)));
}

headerList.add(formatTokenHeaderForSts("Authorization", signature.getAuthorizationHeader()));

// The canonical resource name of the workload identity pool provider.
headerList.add(formatTokenHeaderForSts("x-goog-cloud-target-resource", audience));

GenericJson token = new GenericJson();
token.setFactory(OAuth2Utils.JSON_FACTORY);

token.put("headers", headerList);
token.put("method", signature.getHttpMethod());
token.put(
"url",
((AwsCredentialSource) credentialSource)
.regionalCredentialVerificationUrl.replace("{region}", signature.getRegion()));
return URLEncoder.encode(token.toString(), "UTF-8");
lsirac marked this conversation as resolved.
Show resolved Hide resolved
}

private String getAwsRegion() throws IOException {
// For AWS Lambda, the region is retrieved through the AWS_REGION environment variable.
String region = getEnv("AWS_REGION");
if (region != null) {
return region;
}

AwsCredentialSource awsCredentialSource = (AwsCredentialSource) this.credentialSource;
if (awsCredentialSource.regionUrl == null || awsCredentialSource.regionUrl.isEmpty()) {
throw new IOException(
lsirac marked this conversation as resolved.
Show resolved Hide resolved
"Unable to determine the AWS region. The credential source does not contain the region URL.");
}

region = retrieveResource(awsCredentialSource.regionUrl, "region");

// There is an extra appended character that must be removed. If `us-east-1b` is returned,
// we want `us-east-1`.
return region.substring(0, region.length() - 1);
}

@VisibleForTesting
AwsSecurityCredentials getAwsSecurityCredentials() throws IOException {
// Check environment variables for credentials first.
String accessKeyId = getEnv("AWS_ACCESS_KEY_ID");
String secretAccessKey = getEnv("AWS_SECRET_ACCESS_KEY");
String token = getEnv("Token");
lsirac marked this conversation as resolved.
Show resolved Hide resolved
if (accessKeyId != null && secretAccessKey != null) {
return new AwsSecurityCredentials(accessKeyId, secretAccessKey, token);
}

// Credentials not retrievable from environment variables - call metadata server.
AwsCredentialSource awsCredentialSource = (AwsCredentialSource) credentialSource;
// Retrieve the IAM role that is attached to the VM. This is required to retrieve the AWS
// security credentials.
if (awsCredentialSource.url == null || awsCredentialSource.url.isEmpty()) {
throw new IOException(
"Unable to determine the AWS IAM role name. The credential source does not contain the"
+ " url field.");
}
String roleName = retrieveResource(awsCredentialSource.url, "IAM role");

// Retrieve the AWS security credentials by calling the endpoint specified by the credential
// source.
String awsCredentials =
retrieveResource(awsCredentialSource.url + "/" + roleName, "credentials");

JsonParser parser = OAuth2Utils.JSON_FACTORY.createJsonParser(awsCredentials);
GenericJson genericJson = parser.parseAndClose(GenericJson.class);

accessKeyId = (String) genericJson.get("AccessKeyId");
secretAccessKey = (String) genericJson.get("SecretAccessKey");
token = (String) genericJson.get("Token");

// These credentials last for a few hours - we may consider caching these in the
// future.
return new AwsSecurityCredentials(accessKeyId, secretAccessKey, token);
}

@VisibleForTesting
String getEnv(String name) {
lsirac marked this conversation as resolved.
Show resolved Hide resolved
return System.getenv(name);
}

private static GenericJson formatTokenHeaderForSts(String key, String value) {
// The GCP STS endpoint expects the headers to be formatted as:
// [
// {key: 'x-amz-date', value: '...'},
// {key: 'Authorization', value: '...'},
// ...
// ]
GenericJson header = new GenericJson();
header.setFactory(OAuth2Utils.JSON_FACTORY);
header.put("key", key);
header.put("value", value);
return header;
}

public static AwsCredentials.Builder newBuilder() {
return new AwsCredentials.Builder();
}

public static AwsCredentials.Builder newBuilder(AwsCredentials awsCredentials) {
return new AwsCredentials.Builder(awsCredentials);
}

public static class Builder extends ExternalAccountCredentials.Builder {
lsirac marked this conversation as resolved.
Show resolved Hide resolved

protected Builder() {}
lsirac marked this conversation as resolved.
Show resolved Hide resolved

protected Builder(AwsCredentials credentials) {
super(credentials);
}

@Override
public AwsCredentials build() {
return new AwsCredentials(
transportFactory,
audience,
subjectTokenType,
tokenUrl,
(AwsCredentialSource) credentialSource,
tokenInfoUrl,
serviceAccountImpersonationUrl,
quotaProjectId,
clientId,
clientSecret,
scopes);
}
}
}