Source credentials from the environment

[christophetd]

$ aws-vault exec my-account
$ aws sts get-caller-identity # works
$ awspx ingest
[16/02/23 15:36:14] NOTICE   The profile 'default' doesn't exist. Please enter your AWS credentials.
                             (this information will be saved automatically)
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default output format [None]:

[BViliger18]

Was this resolved?

[Fennerr]

This is covered in the wiki and when you run awspx ingest --help.

Awpx was built so that it runs in a docker container, and the 'awspx' command on your host is just a wrapper to exec into the container. Looking into the awspx file, there is this part that handles the commands:

function awspx(){
    
    if [[ -z "$(docker ps -a -f name=^/awspx$ -q)" ]]; then
        echo -e "[-] Couldn't find \"awspx\" container, you will need to create it first"
        exit 1
    fi

    if [[ -z "$(docker ps -a -f name=^/awspx$ -f status=running -q)" ]]; then
        docker start awspx > /dev/null
    fi

    docker exec -it \
        -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
        -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
        -e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
        -e AWS_SECURITY_TOKEN=$AWS_SECURITY_TOKEN \
        awspx /opt/awspx/cli.py $@
    
}

So it looks like the environment variables are passed through to the container - which is what we want

Starting at 287 in cli.py we get the following:

    pnr.add_argument('--env', action='store_true',
                     help="Use AWS credential environment variables.")
    pnr.add_argument('--profile', dest='profile', default="default",
                     help="Profile to use for ingestion (corresponds to a `[section]` in `~/.aws/credentials).")

So, by default, the --env argument does not have a default value and will not be set, and the default behavior is that the --profile argument is set to default, which is the behavior you are experiencing.

So you need to add the --env flag to awspx ingest to get it to use environment variables. Setting this flag will let this part of the ingest code run:

def handle_ingest(args):
    """
    awspx ingest
    """

    session = None

    # Get credentials from environment variables
    if args.env:
        session = boto3.session.Session(region_name=args.region)

And the boto client will use it's logic to pull the creds from environment variables