Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implements AWS signature version 4 for signing requests #1047

Merged
merged 15 commits into from
Aug 28, 2020
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
280 changes: 280 additions & 0 deletions src/auth/awsrequestsigner.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,280 @@
// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import {GaxiosOptions} from 'gaxios';

import {Headers} from './oauth2client';
import {Crypto, createCrypto, fromArrayBufferToHex} from '../crypto/crypto';

type HttpMethod =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if we use a similar type anywhere else in the codebase, or if gaxios exposes it? If not, no objection to having it here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that gaxios does not export a dedicated type for this.

I couldn't find any similar type in the current repo.

| 'GET'
| 'POST'
| 'PUT'
| 'PATCH'
| 'HEAD'
| 'DELETE'
| 'CONNECT'
| 'OPTIONS'
| 'TRACE';

/** Interface defining the AWS authorization header map for signed requests. */
interface AwsAuthHeaderMap {
amzDate: string;
authorizationHeader: string;
canonicalQuerystring: string;
}

/**
* Interface defining AWS security credentials.
* These are either determined from AWS security_credentials endpoint or
* AWS environment variables.
*/
interface AwsSecurityCredentials {
accessKeyId: string;
secretAccessKey: string;
token?: string;
}

/** AWS Signature Version 4 signing algorithm identifier. */
const AWS_ALGORITHM = 'AWS4-HMAC-SHA256';
/**
* The termination string for the AWS credential scope value as defined in
* https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html
*/
const AWS_REQUEST_TYPE = 'aws4_request';

/**
* Implements an AWS API request signer based on the AWS Signature Version 4
* signing process.
* https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
*/
export class AwsRequestSigner {
private readonly crypto: Crypto;

/**
* Instantiates an AWS API handler used to send authenticated signed
* requests to AWS APIs based on the AWS Signature Version 4 signing process.
* This also provides a mechanism to generate the signed request without
* sending it.
* @param getCredentials A mechanism to retrieve AWS security credentials
* when needed.
* @param region The AWS region to use.
*/
constructor(
private readonly getCredentials: () => Promise<AwsSecurityCredentials>,
private readonly region: string
) {
this.crypto = createCrypto();
}

/**
* Generates the signed request for the provided HTTP request for calling
* an AWS API. This follows the steps described at:
* https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
* @param amzOptions The AWS request options that need to be signed.
* @return A promise that resolves with the GaxiosOptions containing the
* signed HTTP request parameters.
*/
async getRequestOptions(amzOptions: GaxiosOptions): Promise<GaxiosOptions> {
if (!amzOptions.url) {
throw new Error('"url" is required in "amzOptions"');
}
// Stringify JSON requests. This will be set in the request body of the
// generated signed request.
const requestPayloadData =
typeof amzOptions.data === 'object'
? JSON.stringify(amzOptions.data)
: amzOptions.data;
const url = amzOptions.url;
const method = amzOptions.method || 'GET';
const requestPayload = amzOptions.body || requestPayloadData;
const additionalAmzHeaders = amzOptions.headers;
const awsSecurityCredentials = await this.getCredentials();
const uri = new URL(url);
const headerMap = await generateAuthenticationHeaderMap(
this.crypto,
uri.host,
uri.pathname,
uri.search.substr(1),
method,
this.region,
awsSecurityCredentials,
requestPayload,
additionalAmzHeaders
);
// Append additional optional headers, eg. X-Amz-Target, Content-Type, etc.
const headers: {[key: string]: string} = Object.assign(
{
'x-amz-date': headerMap.amzDate,
Authorization: headerMap.authorizationHeader,
host: uri.host,
},
additionalAmzHeaders || {}
);
if (awsSecurityCredentials.token) {
Object.assign(headers, {
'x-amz-security-token': awsSecurityCredentials.token,
});
}
const awsSignedReq: GaxiosOptions = {
url,
method: method,
headers,
};

if (typeof requestPayload !== 'undefined') {
awsSignedReq.body = requestPayload;
}

return awsSignedReq;
}
}

/**
* Creates the HMAC-SHA256 hash of the provided message using the
* provided key.
*
* @param key The HMAC-SHA256 key to use.
* @param msg The message to hash.
* @return The computed hash bytes.
*/
async function sign(
crypto: Crypto,
key: string | ArrayBuffer,
msg: string
): Promise<ArrayBuffer> {
return await crypto.signWithHmacSha256(key, msg);
}

/**
* Calculates the signature for AWS Signature Version 4.
* Based on:
* https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
*
* @param key The AWS secret access key.
* @param dateStamp The '%Y%m%d' date format.
* @param region The AWS region.
* @param serviceName The AWS service name, eg. sts.
* @return The signing key bytes.
*/
async function getSigningKey(
crypto: Crypto,
key: string,
dateStamp: string,
region: string,
serviceName: string
): Promise<ArrayBuffer> {
const kDate = await sign(crypto, `AWS4${key}`, dateStamp);
const kRegion = await sign(crypto, kDate, region);
const kService = await sign(crypto, kRegion, serviceName);
const kSigning = await sign(crypto, kService, 'aws4_request');
return kSigning;
}

/**
* Generates the authentication header map needed for generating the AWS
* Signature Version 4 signed request.
*
* @param accessKeyId The AWS access key ID.
* @param secretAccessKey The AWS secret access kye.
* @param token The AWS token.
* @return The AWS authentication header map which constitutes of the following
* components: amz-date, authorization header and canonical query string.
*/
async function generateAuthenticationHeaderMap(
crypto: Crypto,
host: string,
canonicalUri: string,
canonicalQuerystring: string,
method: HttpMethod,
region: string,
securityCredentials: AwsSecurityCredentials,
requestPayload = '',
additionalAmzHeaders: Headers = {}
): Promise<AwsAuthHeaderMap> {
// iam.amazonaws.com host => iam service.
// sts.us-east-2.amazonaws.com => sts service.
const serviceName = host.split('.')[0];
const now = new Date();
// Format: '%Y%m%dT%H%M%SZ'.
const amzDate = now
.toISOString()
.replace(/[-:]/g, '')
.replace(/\.[0-9]+/, '');
// Format: '%Y%m%d'.
const dateStamp = now.toISOString().replace(/[-]/g, '').replace(/T.*/, '');

// Change all additional headers to be lower case.
const reformattedAdditionalAmzHeaders: Headers = {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is definitely the most complex I think I've bumped into during this review, I believe OAuth 1.0 worked similarly, with regards to needing to sort and sign parameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree. The crypto they use is not trivial and it is not standard. You are right on the similarities with OAuth 1.0 but at least the latter is a standard and there is sufficient documentation on it.
We had to follow the AWS documentation in order to generate these (step by step). They do provide reference input/outputs for each step which helped.

I think the bulk of the logic is documented in generateAuthenticationHeaderMap. I provided links to the 4 steps and some comments to clarify intermediate steps. I hope this makes it easier for contributors to follow. One possibility that was floated around in the early stages was to pull in AWS SDK as a dependency. However, I think it would be too expensive for serverless and low memory environments like GCF, IOT devices, etc. The auth library is a dependency for a lot of other libraries where this feature is not used so it could be very taxing to these users.

Object.keys(additionalAmzHeaders).forEach(key => {
reformattedAdditionalAmzHeaders[key.toLowerCase()] =
additionalAmzHeaders[key];
});
// Add AWS token if available.
if (securityCredentials.token) {
reformattedAdditionalAmzHeaders['x-amz-security-token'] =
securityCredentials.token;
}
// Header keys need to be sorted alphabetically.
const amzHeaders = Object.assign(
{
host,
'x-amz-date': amzDate,
},
reformattedAdditionalAmzHeaders
);
let canonicalHeaders = '';
const signedHeadersList = Object.keys(amzHeaders).sort();
signedHeadersList.forEach(key => {
canonicalHeaders += `${key}:${amzHeaders[key]}\n`;
});
const signedHeaders = signedHeadersList.join(';');

const payloadHash = await crypto.sha256DigestHex(requestPayload);
// https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
const canonicalRequest =
`${method}\n` +
`${canonicalUri}\n` +
`${canonicalQuerystring}\n` +
`${canonicalHeaders}\n` +
`${signedHeaders}\n` +
`${payloadHash}`;
const credentialScope = `${dateStamp}/${region}/${serviceName}/${AWS_REQUEST_TYPE}`;
// https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html
const stringToSign =
`${AWS_ALGORITHM}\n` +
`${amzDate}\n` +
`${credentialScope}\n` +
(await crypto.sha256DigestHex(canonicalRequest));
// https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
const signingKey = await getSigningKey(
crypto,
securityCredentials.secretAccessKey,
dateStamp,
region,
serviceName
);
const signature = await sign(crypto, signingKey, stringToSign);
// https://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html
const authorizationHeader =
`${AWS_ALGORITHM} Credential=${securityCredentials.accessKeyId}/` +
`${credentialScope}, SignedHeaders=${signedHeaders}, ` +
`Signature=${fromArrayBufferToHex(signature)}`;

return {
amzDate,
authorizationHeader,
canonicalQuerystring,
};
}
9 changes: 9 additions & 0 deletions test/fixtures/aws-security-credentials.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"Code" : "Success",
"LastUpdated" : "2020-08-11T19:33:07Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIARD4OQDT6A77FR3CL",
"SecretAccessKey" : "Y8AfSaucF37G4PpvfguKZ3/l7Id4uocLXxX0+VTx",
"Token" : "IQoJb3JpZ2luX2VjEIz//////////wEaCXVzLWVhc3QtMiJGMEQCIH7MHX/Oy/OB8OlLQa9GrqU1B914+iMikqWQW7vPCKlgAiA/Lsv8Jcafn14owfxXn95FURZNKaaphj0ykpmS+Ki+CSq0AwhlEAAaDDA3NzA3MTM5MTk5NiIMx9sAeP1ovlMTMKLjKpEDwuJQg41/QUKx0laTZYjPlQvjwSqS3OB9P1KAXPWSLkliVMMqaHqelvMF/WO/glv3KwuTfQsavRNs3v5pcSEm4SPO3l7mCs7KrQUHwGP0neZhIKxEXy+Ls//1C/Bqt53NL+LSbaGv6RPHaX82laz2qElphg95aVLdYgIFY6JWV5fzyjgnhz0DQmy62/Vi8pNcM2/VnxeCQ8CC8dRDSt52ry2v+nc77vstuI9xV5k8mPtnaPoJDRANh0bjwY5Sdwkbp+mGRUJBAQRlNgHUJusefXQgVKBCiyJY4w3Csd8Bgj9IyDV+Azuy1jQqfFZWgP68LSz5bURyIjlWDQunO82stZ0BgplKKAa/KJHBPCp8Qi6i99uy7qh76FQAqgVTsnDuU6fGpHDcsDSGoCls2HgZjZFPeOj8mmRhFk1Xqvkbjuz8V1cJk54d3gIJvQt8gD2D6yJQZecnuGWd5K2e2HohvCc8Fc9kBl1300nUJPV+k4tr/A5R/0QfEKOZL1/k5lf1g9CREnrM8LVkGxCgdYMxLQow1uTL+QU67AHRRSp5PhhGX4Rek+01vdYSnJCMaPhSEgcLqDlQkhk6MPsyT91QMXcWmyO+cAZwUPwnRamFepuP4K8k2KVXs/LIJHLELwAZ0ekyaS7CptgOqS7uaSTFG3U+vzFZLEnGvWQ7y9IPNQZ+Dffgh4p3vF4J68y9049sI6Sr5d5wbKkcbm8hdCDHZcv4lnqohquPirLiFQ3q7B17V9krMPu3mz1cg4Ekgcrn/E09NTsxAqD8NcZ7C7ECom9r+X3zkDOxaajW6hu3Az8hGlyylDaMiFfRbBJpTIlxp7jfa7CxikNgNtEKLH9iCzvuSg2vhA==",
"Expiration" : "2020-08-11T07:35:49Z"
}