cannot recognize global credential and s3 arn

[joyfangcheng]

hello all,

i'm following this guidance https://github.com/awslabs/data-transfer-hub/blob/main/docs/S3_PLUGIN.md#data-transfer-hub---s3-plugin to deploy s3 plugin in AWS China region, to transfer objects from China to global. Source bucket is located in AWS China, destination bucket is located in AWS Ireland.

After deployment has finished, i notice that in FinderStackFinderRoleA666A5A8, destCredential and destBucket arn cannot be recognized as global arn, I tried to change it to global arn, but obviously it failed :( Would like to know if here's anyone getting same requirement and how to solve it?

Thanks a lot,
Joy FANG

[YikaiHu]

Hi @joyfangcheng , could you provide your Cloudformation Parameters?
And please follow our latest documentation: https://awslabs.github.io/data-transfer-hub/en/user-guide/tutorial-s3/#using-the-s3-plugin

[joyfangcheng]

Hello @YikaiHu,

Thanks a lot for your reply, please check following CF parameters:

I tried with latest stack again, but got the same issue, destCredential could not be recognized as global arn, screenshot in DefaultPolicyBB6BAB25
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": [
"arn:aws-cn:secretsmanager:cn-northwest-1:528369405573:secret:dth-src-credential-??????",
"arn:aws-cn:secretsmanager:cn-northwest-1:528369405573:secret:landingzone-China-logs-credential-??????"
],
"Effect": "Allow"
}

error in cloudwatch:
2024/05/03 03:43:53 Unable to list objects in / - operation error S3: ListObjectsV2, https response error StatusCode: 400, RequestID: 2X2J1S6VEHGE66CX, HostID: c5uXnKsGp5PP4sHYs72ml4GjlgegOu5Wih35j14xvpyun4GjwwZBqyLiaubF6AXWpKhmXyC/FzM=, api error InvalidToken: The provided token is malformed or otherwise invalid.

Any comments would be really apprciated.

B.R.
Joy FANG

[YikaiHu]

Hi @joyfangcheng ,
The information for the region‘s ARN in the credentials provided in your DefaultPolicyBB6BAB25 is correct. This is because it is the ARN of the Secrets where your Access Key and Secrets Key are stored, which are both located in the same region where you deployed your DTH.

So I have two questions may be can find out the root cause:

1. It appears that both your S3 source and S3 destination are not in the same account as your DTH deployment. Is this correct? So, the actual setup is: Source S3 (in Account A - cn-northwest-1) -> DTH (in Account B - cn-northwest-1) -> Destination S3 (in Account C - eu-west-1), correct?
2. If the answer of question 1 is correct, please check the format of your secret stored in AWS Secrets Manager. Your secret must adhere to the following format:

{
  "access_key_id": "<Your Access Key ID>",
  "secret_access_key": "<Your Access Key Secret>"
}

[joyfangcheng]

hi @YikaiHu ,

Sorry for my late response. Thanks for your help.

i have solved the ak/sk issue. As destCredential is located in global AWS region, i created ak/sk in China region with global ak/sk value, then EC2 finder can recognize.

But i got another issue, failed to list object in source bucket:
2024/05/08 06:22:47 Fail to get source list - operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: JZ7YWBJMKDR0BET0, HostID: K9Xymy8n8ZXOzE1UwtyubVv7LUWfKSyDu+bbc6Ery2V0MXjv3M1Jp6TAlF9rSSi51imIH+iwypY=, api error AccessDenied: Access Denied

I saw here's the description in template: "Source prefix list file S3 bucket name (Optional). It used to store the Source prefix list file. The specified bucket must be located in the same AWS region and under the same account as the DTH deployment". but my DTH stack and srcBucket are running in different AWS China accounts, is it because of this which results in the error? if i want to keep current 2 different accounts for DTH deployment and source Bucket, do i need to define PrefixList in source bucket first?

Thanks a lot,
Joy FANG

[YikaiHu]

Oh I see the root cause.
If you do not need to use source prefixlist feature, you can just leave SourcePrefixListBucket empty.

[joyfangcheng]

before i left it as blank, i got the error:

Error: waiting for CloudFormation Stack (arn:aws-cn:cloudformation:cn-northwest-1:737033023347:stack/DTHS3Stack/c54b52a0-0164-11ef-8a6f-026334cbc2ba) create: failed to create CloudFormation stack, rollback requested (ROLLBACK_COMPLETE): ["The following resource(s) failed to create: [EC2WorkerStackWorkerEC2LaunchTemplateProfile62CB513A, FinderStackFinderEC2LaunchTemplateProfileC751664D, EC2WorkerStackWorkerAsgRoleDefaultPolicy3BF8DE22, FinderStackFinderRoleDefaultPolicy820CDD13]. Rollback requested by user." "Resource creation cancelled" "Resource creation cancelled" "Resource creation cancelled" "Resource handler returned message: "Invalid policy (Service: Iam, Status Code: 400, Request ID: 6e767d94-3005-4a3d-aa13-d1ba7da67ff4)" (RequestToken: 3600f1ee-49d0-6ba7-5a99-532cae2b8ffe, HandlerErrorCode: InvalidRequest)"]

then i added value, it can continue creating stack

[YikaiHu]

@joyfangcheng

It's wired, I leave it blank and the creation is successful.

Below is my parameters:

And the template link is : https://solutions-reference.s3.amazonaws.com/data-transfer-hub/latest/DataTransferS3Stack.template

[joyfangcheng]

hi @YikaiHu,

Thanks for the information, I used same same template you shared, but got the error on role policy :
Resource handler returned message: "Invalid policy (Service: Iam, Status Code: 400, Request ID: a0ac7f5d-a42c-414d-9db8-39508c4bc78c)" (RequestToken: 08d986c5-b1c0-f45d-1771-8b663e036afb, HandlerErrorCode: InvalidRequest)

getting the same error, when trying to leave srcPrefixListBucket with empty and updating my previous stack.

policy details as below:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "cloudformation:SignalResource",
"Resource": "arn:aws-cn:cloudformation:cn-northwest-1:737033023347:stack/DTHS3Stack/df118c60-06f8-11ef-b24f-02cd875d1578",
"Effect": "Allow"
},
{
"Action": [
"sqs:SendMessage",
"sqs:GetQueueAttributes",
"sqs:GetQueueUrl"
],
"Resource": "arn:aws-cn:sqs:cn-northwest-1:737033023347:DTHS3Stack-S3TransferQueue-iJvTFlnBvCqi",
"Effect": "Allow"
},
{
"Action": [
"s3:GetObject*",
"s3:GetBucket*",
"s3:List*"
],
"Resource": [
"arn:aws-cn:s3:::landingzone-china-logs-737033023347-cn-northwest-1",
"arn:aws-cn:s3:::landingzone-china-logs-737033023347-cn-northwest-1/"
],
"Effect": "Allow"
},
{
"Action": [
"s3:GetObject",
"s3:GetBucket*",
"s3:List*"
],
"Resource": [
"arn:aws-cn:s3:::landingzone-china-logs-332352433694-eu-west-1",
"arn:aws-cn:s3:::landingzone-china-logs-332352433694-eu-west-1/"
],
"Effect": "Allow"
},
{
"Action": [
"s3:GetObject",
"s3:GetBucket*",
"s3:List*"
],
"Resource": [
"arn:aws-cn:s3:::landingzone-china-logs-737033023347-cn-northwest-1",
"arn:aws-cn:s3:::landingzone-china-logs-737033023347-cn-northwest-1/"
],
"Effect": "Allow"
},
{
"Action": [
"states:ListExecutions",
"states:ListStateMachines"
],
"Resource": "arn:aws-cn:states:cn-northwest-1:737033023347:stateMachine:DTHS3Stack-MultiPart-ControllerSM",
"Effect": "Allow"
},
{
"Action": [
"states:DescribeExecution",
"states:DescribeStateMachineForExecution",
"states:GetExecutionHistory"
],
"Resource": "arn:aws-cn:states:cn-northwest-1:737033023347:execution:DTHS3Stack-MultiPart-ControllerSM:",
"Effect": "Allow"
},
{
"Action": [
"states:ListActivities",
"states:DescribeStateMachine",
"states:DescribeActivity"
],
"Resource": "*",
"Effect": "Allow"
}
]
}

[YikaiHu]

@joyfangcheng , we have finded out the root cause.

This failure is due to changes in the underlying IAM logic implemented by AWS. As a result, this issue may occur in new accounts but not in my old account.

Please using the following patch template to create the Transfer Task, and please leave SourcePrefixListBucket blank.

DataTransferS3Stack-fix-0509.json

[YikaiHu]

We will plan a patch in next version

[joyfangcheng]

Thanks a lot for your help @YikaiHu my issue has been fixed.

B.R.
Joy FANG

[YikaiHu]

Hi @joyfangcheng , we noticed that you have posted an potential AWS AKSK in github.
This credential is no longer secure.
Please deactivate it as soon as possible!!