vdb-config AWS configuration challenge

[tweep]

Hi there,
I'm using prefetch version 3.0.0. and I'm trying to download some dbGAP data. within vdb-config, I have selected

• remote access (main tab)
• accept charges (aws tab)

I also read trough https://github.com/ncbi/sra-tools/wiki/04.-Cloud-Credentials which describes, how the AWS IAM credentials can be retrieved. I've tried to format the credentials in various ways, ie pointing to to the downloaded csv file, changing read permissions etc -- however when I try to run prefetch it fails with this error:

2022-02-26T02:12:17 prefetch.3.0.0 int: file unauthorized while opening file within network system module - cannot open remote file: https://sra-ca-run-odp.s3.amazonaws.com/sra/phs001963.c1/SRR15467117/SRR15467117

Interestingly, when deselecting the "accept charges" option in AWS tab of vdb-config, the download works OK.
This seems to indicate that the authentication with the AWS IAM credentials is the issue here. I'm trying to use AWS as I prefer to download directly from S3, rather than via the slower https.

Questions:

• Can you please update the documentation on how to configure the AWS credentials file used by vdb-config and provide an example format ? ( the standard AWS credentials format seems to not work for me )
• The vdb-config ui does not allow to select specific files - how will the config identify the correct file ? does it have to be named in a certain way ? (would be great if the Amazon cloud credentials page gets updated)
• Does the IAM user, which I am using, require certain permissions ?

[skripche]

Did you select "report cloud instance identity"?

Were you using an NGC file or JWT cart to access the dbGaP data?

[zhangguy]

Did you select "report cloud instance identity"?

Were you using an NGC file or JWT cart to access the dbGaP data?

I encountered the same error as the original poster, no matter "report cloud instance identity" is checked or not. I tried both NGC file and the JWT cart file and got the same error. I'm using the latest version of sratools. Right now I can only download from the sra repo but not from the s3. Could you help? Thanks

[stineaj]

@zhangguy please write to sra@ncbi.nlm.nih.gov to open a ticket. We will want to ask you to provide some additional details for the exact commands and configuration you are using and some of the output may provide more info than I am comfortable asking you to post in a public forum.