feat: implements AWS signature version 4 for signing requests #1047

bojeil-google · 2020-08-24T21:31:20Z

This is based on:
https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html

It will be used to generate signed requests to AWS GetCallerIdentity API.
https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html

This API is used to return details about the IAM user or role whose
credentials are used to call the operation.

* chore: updated samples/package.json [ci skip] * chore: updated CHANGELOG.md [ci skip] * chore: updated package.json Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/5f7f9c6d-c75a-4c60-8bb8-0026a14cead7/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@94421c4

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/b742586e-df31-4aac-8092-78288e9ea8e7/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@bd0deaa

This PR was generated using Autosynth. 🌈 - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@5747555

* chore: updated samples/package.json [ci skip] * chore: updated CHANGELOG.md [ci skip] * chore: updated package.json Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Minor change that fixes a spelling error. I don't think an issue needed to be opened for this. Thanks!

Add await Co-authored-by: Justin Beckwith <justin.beckwith@gmail.com> Co-authored-by: sofisl <55454395+sofisl@users.noreply.github.com> Co-authored-by: Benjamin E. Coe <bencoe@google.com>

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/7a1b0b96-8ddb-4836-a1a2-d2f73b7e6ffe/targets - [ ] To automatically regenerate this PR, check this box.

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/ba2d388f-b3b2-4ad7-a163-0c6b4d86894f/targets - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@05de3e1

This is based on: https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html It will be used to generate signed requests to AWS GetCallerIdentity API. https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html This API is used to return details about the IAM user or role whose credentials are used to call the operation.

codecov · 2020-08-24T21:31:27Z

Codecov Report

❗ No coverage uploaded for pull request base (byoid@26ab5e3). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             byoid    #1047   +/-   ##
========================================
  Coverage         ?   92.90%           
========================================
  Files            ?       26           
  Lines            ?     5528           
  Branches         ?      613           
========================================
  Hits             ?     5136           
  Misses           ?      392           
  Partials         ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26ab5e3...80e3f92. Read the comment docs.

google-cla · 2020-08-24T21:31:33Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

bcoe

Reading through this, I definitely am left wishing there was less complexity 😆

But I also think the implementation is sound, and doesn't introduce any unnecessary complexity.

bcoe · 2020-08-27T01:23:30Z

src/auth/awsrequestsigner.ts

+import {Headers} from './oauth2client';
+import {Crypto, createCrypto, fromArrayBufferToHex} from '../crypto/crypto';
+
+type HttpMethod =


Curious if we use a similar type anywhere else in the codebase, or if gaxios exposes it? If not, no objection to having it here.

It seems that gaxios does not export a dedicated type for this.

I couldn't find any similar type in the current repo.

bcoe · 2020-08-27T01:28:38Z

src/auth/awsrequestsigner.ts

+  const dateStamp = now.toISOString().replace(/[-]/g, '').replace(/T.*/, '');
+
+  // Change all additional headers to be lower case.
+  const reformattedAdditionalAmzHeaders: Headers = {};


This logic is definitely the most complex I think I've bumped into during this review, I believe OAuth 1.0 worked similarly, with regards to needing to sort and sign parameters?

I completely agree. The crypto they use is not trivial and it is not standard. You are right on the similarities with OAuth 1.0 but at least the latter is a standard and there is sufficient documentation on it.
We had to follow the AWS documentation in order to generate these (step by step). They do provide reference input/outputs for each step which helped.

I think the bulk of the logic is documented in generateAuthenticationHeaderMap. I provided links to the 4 steps and some comments to clarify intermediate steps. I hope this makes it easier for contributors to follow. One possibility that was floated around in the early stages was to pull in AWS SDK as a dependency. However, I think it would be too expensive for serverless and low memory environments like GCF, IOT devices, etc. The auth library is a dependency for a lot of other libraries where this feature is not used so it could be very taxing to these users.

bcoe · 2020-08-27T01:31:12Z

test/test.awsrequestsigner.ts

+    // List of various requests and their expected signatures.
+    // Examples source:
+    // https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
+    const getRequestOptionsTests: AwsRequestSignerTest[] = [


I like this DSL you've built out for exercising the the fairly complex logic in the AWS request signer.

Thank you. I think this simplifies adding and removing new tests for this logic. It also makes it easier to replace the existing canonical request/signatures we are using. So it should not be expensive to maintain and build on in the future.

https://github.com/boto/botocore/tree/879f8440a4e9ace5d3cf145ce8b3d5e5ffb892ef/tests/unit/auth/aws4_testsuite Adds support for raw date header in AWS requests. Fixes padding issue in `fromArrayBufferToHex`.

google-cla · 2020-08-28T00:30:41Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

bcoe · 2020-08-28T02:33:43Z

test/test.awsrequestsigner.ts

+            headers: {
+              Authorization:
+                'AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20110909/us-east-1/host/' +
+                `aws4_request, SignedHeaders=date;host;zoo, Signature=${signature}`,


thank you for pulling these tests over from boto 👍 sorry for being a pain.

feat: implements the OAuth token exchange spec based on rfc8693 (#1026) feat: defines ExternalAccountClient abstract class for external_account credentials (#1030) feat: adds service account impersonation to `ExternalAccountClient` (#1041) feat: defines `IdentityPoolClient` used for K8s and Azure workloads (#1042) feat: implements AWS signature version 4 for signing requests (#1047) feat: defines `ExternalAccountClient` used to instantiate external account clients (#1050) feat!: integrates external_accounts with `GoogleAuth` and ADC (#1052) feat: adds text/json credential_source support to IdentityPoolClients (#1059) feat: get AWS region from environment variable (#1067) Co-authored-by: Wilfred van der Deijl <wilfred@vanderdeijl.com> Co-authored-by: Benjamin E. Coe <bencoe@google.com>

grayside and others added 14 commits August 7, 2020 16:32

refactor(samples): idtoken-cloudrun => idtoken-serverless (#1025)

11b0cb7

docs: README updated with new sample paths

4df48df

chore: release 6.0.7 (#1023)

6dfee1d

* chore: updated samples/package.json [ci skip] * chore: updated CHANGELOG.md [ci skip] * chore: updated package.json Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

fix(deps): roll back dependency google-auth-library to ^6.0.6 (#1033)

eb54ee9

build: perform publish using Node 12 (#1035)

714247d

This PR was generated using Autosynth. 🌈 - [ ] To automatically regenerate this PR, check this box. Source-Link: googleapis/synthtool@5747555

chore: release 6.0.8 (#1036)

211e042

* chore: updated samples/package.json [ci skip] * chore: updated CHANGELOG.md [ci skip] * chore: updated package.json Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

docs: correct spelling of its (#1040)

d4e56c0

Minor change that fixes a spelling error. I don't think an issue needed to be opened for this. Thanks!

docs: fix usage in readme (#1031)

6c15139

Add await Co-authored-by: Justin Beckwith <justin.beckwith@gmail.com> Co-authored-by: sofisl <55454395+sofisl@users.noreply.github.com> Co-authored-by: Benjamin E. Coe <bencoe@google.com>

chore: start tracking obsolete files (#1043)

0c8e086

This PR was generated using Autosynth. 🌈 Synth log will be available here: https://source.cloud.google.com/results/invocations/7a1b0b96-8ddb-4836-a1a2-d2f73b7e6ffe/targets - [ ] To automatically regenerate this PR, check this box.

Merge remote-tracking branch 'upstream/byoid'

2cd3f93

bojeil-google requested a review from a team as a code owner August 24, 2020 21:31

blunderbuss-gcf bot assigned sofisl Aug 24, 2020

google-cla bot added the cla: no This human has *not* signed the Contributor License Agreement. label Aug 24, 2020

bcoe approved these changes Aug 27, 2020

View reviewed changes

bcoe added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Aug 27, 2020

Adds additional AWS signature v4 tests from AWS botocore tests.

80e3f92

https://github.com/boto/botocore/tree/879f8440a4e9ace5d3cf145ce8b3d5e5ffb892ef/tests/unit/auth/aws4_testsuite Adds support for raw date header in AWS requests. Fixes padding issue in `fromArrayBufferToHex`.

google-cla bot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Aug 28, 2020

bcoe added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Aug 28, 2020

bcoe reviewed Aug 28, 2020

View reviewed changes

bcoe approved these changes Aug 28, 2020

View reviewed changes

bcoe merged commit 3020c6f into googleapis:byoid Aug 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implements AWS signature version 4 for signing requests #1047

feat: implements AWS signature version 4 for signing requests #1047

bojeil-google commented Aug 24, 2020

codecov bot commented Aug 24, 2020 •

edited

google-cla bot commented Aug 24, 2020

bcoe left a comment

bcoe Aug 27, 2020

bojeil-google Aug 27, 2020

bcoe Aug 27, 2020

bojeil-google Aug 27, 2020

bcoe Aug 27, 2020

bojeil-google Aug 27, 2020

google-cla bot commented Aug 28, 2020

bcoe Aug 28, 2020

feat: implements AWS signature version 4 for signing requests #1047

feat: implements AWS signature version 4 for signing requests #1047

Conversation

bojeil-google commented Aug 24, 2020

codecov bot commented Aug 24, 2020 • edited

Codecov Report

google-cla bot commented Aug 24, 2020

bcoe left a comment

Choose a reason for hiding this comment

bcoe Aug 27, 2020

Choose a reason for hiding this comment

bojeil-google Aug 27, 2020

Choose a reason for hiding this comment

bcoe Aug 27, 2020

Choose a reason for hiding this comment

bojeil-google Aug 27, 2020

Choose a reason for hiding this comment

bcoe Aug 27, 2020

Choose a reason for hiding this comment

bojeil-google Aug 27, 2020

Choose a reason for hiding this comment

google-cla bot commented Aug 28, 2020

bcoe Aug 28, 2020

Choose a reason for hiding this comment

codecov bot commented Aug 24, 2020 •

edited