Permission denied (publickey) on EC2 cloud

[Tupoun]

OS / Environment

Ubuntu 16.04.2 LTS

Ansible version

ansible 2.0.0.2

Version of components from requirements.txt

Name: msrestazure
Version: 0.4.7
Summary: AutoRest swagger generator Python client runtime. Azure-specific module.
Home-page: https://github.com/Azure/msrestazure-for-python
Author: Microsoft Corporation
Author-email: UNKNOWN
License: MIT License
Location: /mnt/backup/Install/algo-master/env/lib/python2.7/site-packages
Requires: keyring, msrest, adal
---
Name: boto3
Version: 1.4.4
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /mnt/backup/Install/algo-master/env/lib/python2.7/site-packages
Requires: s3transfer, jmespath, botocore
---
Name: apache-libcloud
Version: 1.5.0
Summary: A standard Python library that abstracts away differences among multiple cloud provider APIs. For more information and documentation, please see http://libcloud.apache.org
Home-page: http://libcloud.apache.org/
Author: Apache Software Foundation
Author-email: dev@libcloud.apache.org
License: Apache License (2.0)
Location: /mnt/backup/Install/algo-master/env/lib/python2.7/site-packages
Requires: 
---
Name: six
Version: 1.10.0
Summary: Python 2 and 3 compatibility utilities
Home-page: http://pypi.python.org/pypi/six/
Author: Benjamin Peterson
Author-email: benjamin@python.org
License: MIT
Location: /mnt/backup/Install/algo-master/env/lib/python2.7/site-packages
Requires: 
---
Name: pyOpenSSL
Version: 16.2.0
Summary: Python wrapper module around the OpenSSL library
Home-page: https://pyopenssl.readthedocs.io/
Author: Hynek Schlawack
Author-email: hs@ox.cx
License: Apache License, Version 2.0
Location: /mnt/backup/Install/algo-master/env/lib/python2.7/site-packages
Requires: cryptography, six

Summary of the problem

Instalation to the EC2 end with error

fatal: [35.156.181.105]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '35.156.181.105' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}

Steps to reproduce the behavior

Step 1 to 6 from Deploy the Algo server

The way of deployment (cloud or local)

Cloud EC2

Expected behavior

Installed

Actual behavior

Not Installed

Full log

  What provider would you like to use?
    1. DigitalOcean
    2. Amazon EC2
    3. Microsoft Azure
    4. Google Compute Engine (only for testing, see issue #369)
    5. Install to existing Ubuntu 16.04 server

Enter the number of your desired provider
: 2

Enter your aws_access_key (http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
Note: Make sure to use either your root key (recommended) or an IAM user with an acceptable policy attached
[pasted values will not be displayed]
[AKIA...]: 

Enter your aws_secret_key (http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
Note: Make sure to use either your root key (recommended) or an IAM user with an acceptable policy attached
[pasted values will not be displayed]
[ABCD...]: 

Name the vpn server:
[algo]: 


  What region should the server be located in?
    1.   us-east-1           US East (N. Virginia)
    2.   us-east-2           US East (Ohio)
    3.   us-west-1           US West (N. California)
    4.   us-west-2           US West (Oregon)
    5.   ap-south-1          Asia Pacific (Mumbai)
    6.   ap-northeast-2      Asia Pacific (Seoul)
    7.   ap-southeast-1      Asia Pacific (Singapore)
    8.   ap-southeast-2      Asia Pacific (Sydney)
    9.   ap-northeast-1      Asia Pacific (Tokyo)
    10.  eu-central-1        EU (Frankfurt)
    11.  eu-west-1           EU (Ireland)
    12.  eu-west-2           EU (London)
    13.  ca-central-1        Canada (Central)
Enter the number of your desired region:
[1]: 10

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
[y/N]: y

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
[y/N]: y

List the names of trusted Wi-Fi networks (if any) that macOS/iOS clients exclude from using the VPN (e.g., your home network. Comma-separated value, e.g., HomeNet,OfficeWifi,AlgoWiFi)
: 

Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
[y/N]: y

Do you want each user to have their own account for SSH tunneling?
[y/N]: y

Do you want to apply operating system security enhancements on the server? (warning: replaces your sshd_config)
[y/N]: 

Do you want the VPN to support Windows 10 clients? (requires RSA certificates and key exchange, less secure)
[y/N]: y

Do you want to retain the CA key? (required to add users in the future, but less secure)
[y/N]: 

PLAY [Configure the server] ****************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [Generate the SSH private key] ********************************************
ok: [localhost -> localhost]

TASK [Generate the SSH public key] *********************************************
ok: [localhost -> localhost]

TASK [Change mode for the SSH private key] *************************************
ok: [localhost -> localhost]

TASK [Ensure the dynamic inventory exists] *************************************
ok: [localhost]

TASK [cloud-ec2 : Locate official Ubuntu 16.04 AMI for region] *****************
ok: [localhost]

TASK [cloud-ec2 : set_fact] ****************************************************
ok: [localhost]

TASK [cloud-ec2 : Add ssh public key] ******************************************
ok: [localhost] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGSaI4POiDn+Z336Ybg38OBmGhsIEUKOArOL+CFHpCGOIMF3I2pe0g7vN0usClTkNBLeALFEOE/u8x3uIZ6IsV281Q4HRvJGV/6LmAeu/s7Ifk8GjFirV7BRx5mrUROBAuhUrEIlbyeP2QN7MrAnwuDrvElA+oPterD7uMInYhSlrta9tJ0U3FYSRHijRrGsthW0J4jedyPiGIezlSoS3qWwiqEpD5/DG4fsj32hyGJGAtVBusprBxuapTE1Op0VBrZrIt/E9KFTG9I+callmJjg0sNmOUjdFzR3CnO54F0jqBIIxFqV1Y4DlVpoDHxizRgEByF7EfIE6oWz2expLX algo@ssh)                                                                                                                                                                                               

TASK [cloud-ec2 : Configure EC2 virtual private clouds] ************************
ok: [localhost]

TASK [cloud-ec2 : Set up Public Subnets Route Table] ***************************
ok: [localhost]

TASK [cloud-ec2 : Configure EC2 security group] ********************************
ok: [localhost]

TASK [cloud-ec2 : Launch instance] *********************************************
ok: [localhost]

TASK [cloud-ec2 : Add new instance to host group] ******************************
changed: [localhost] => (item={u'kernel': None, u'root_device_type': u'ebs', u'private_dns_name': u'ip-172-16-254-49.eu-central-1.compute.internal', u'public_ip': u'35.156.181.105', u'private_ip': u'172.16.254.49', u'id': u'i-0fc78cbab502b962d', u'ebs_optimized': False, u'state': u'running', u'virtualization_type': u'hvm', u'architecture': u'x86_64', u'ramdisk': None, u'block_device_mapping': {u'/dev/sda1': {u'status': u'attached', u'delete_on_termination': True, u'volume_id': u'vol-0851e296efc14f3c9'}}, u'key_name': u'VPNKEY', u'image_id': u'ami-060cde69', u'tenancy': u'default', u'groups': {u'sg-09e94362': u'vpn-secgroup'}, u'public_dns_name': u'ec2-35-156-181-105.eu-central-1.compute.amazonaws.com', u'state_code': 16, u'tags': {u'Environment': u'Algo', u'Name': u'algo'}, u'placement': u'eu-central-1b', u'ami_launch_index': u'0', u'dns_name': u'ec2-35-156-181-105.eu-central-1.compute.amazonaws.com', u'region': u'eu-central-1', u'launch_time': u'2017-04-14T20:07:09.000Z', u'instance_type': u't2.micro', u'root_device_name': u'/dev/sda1', u'hypervisor': u'xen'})

TASK [cloud-ec2 : set_fact] ****************************************************
ok: [localhost]

TASK [cloud-ec2 : Get EC2 instances] *******************************************
ok: [localhost]

TASK [cloud-ec2 : Ensure the group ec2 exists in the dynamic inventory file] ***
ok: [localhost]

TASK [cloud-ec2 : Populate the dynamic inventory] ******************************
ok: [localhost] => (item={u'kernel': None, u'instance_profile': None, u'root_device_type': u'ebs', u'private_dns_name': u'ip-172-16-254-49.eu-central-1.compute.internal', u'spot_instance_request_id': None, u'source_destination_check': u'true', u'id': u'i-0fc78cbab502b962d', u'ebs_optimized': False, u'state': u'running', u'client_token': u'', u'virtualization_type': u'hvm', u'ramdisk': None, u'public_ip_address': u'35.156.181.105', u'block_device_mapping': [{u'status': u'attached', u'volume_id': u'vol-0851e296efc14f3c9', u'delete_on_termination': True, u'attach_time': u'2017-04-14T20:07:09.000Z', u'device_name': u'/dev/sda1'}], u'key_name': u'VPNKEY', u'interfaces': [{u'id': u'eni-d59bcaab', u'mac_address': u'06:28:22:45:fe:9b'}], u'image_id': u'ami-060cde69', u'groups': [{u'id': u'sg-09e94362', u'name': u'vpn-secgroup'}], u'public_dns_name': u'ec2-35-156-181-105.eu-central-1.compute.amazonaws.com', u'requester_id': None, u'tags': {u'Environment': u'Algo', u'Name': u'algo'}, u'monitoring_state': u'disabled', u'placement': {u'tenancy': u'default', u'zone': u'eu-central-1b'}, u'ami_launch_index': u'0', u'hypervisor': u'xen', u'region': u'eu-central-1', u'launch_time': u'2017-04-14T20:07:09.000Z', u'persistent': False, u'architecture': u'x86_64', u'private_ip_address': u'172.16.254.49', u'vpc_id': u'vpc-8e74fae6'})

TASK [Wait until SSH becomes ready...] *****************************************
ok: [localhost -> localhost]

TASK [A short pause, in order to be sure the instance is ready] ****************
Pausing for 10 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [localhost]

TASK [Ensure the local ssh directory is exist] *********************************
ok: [localhost -> localhost]

TASK [Copy the algo ssh key to the local ssh directory] ************************
ok: [localhost -> localhost]

TASK [Configure the local ssh config] ******************************************
ok: [localhost -> localhost]

PLAY [Configure the server and install required software] **********************

TASK [Check the system] ********************************************************
fatal: [35.156.181.105]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '35.156.181.105' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}

PLAY RECAP *********************************************************************
35.156.181.105             : ok=0    changed=0    unreachable=1    failed=0   
localhost                  : ok=22   changed=1    unreachable=0    failed=0  

[jackivanov]

ansible 2.0.0.2

Did you follow the README and install the requirements?

[Tupoun]

Yes, I did it. I think I used wrong command to list it ( pip show $(< requirements.txt) ). So the complete list is below (created by pip list):

adal (0.4.5)
ansible (2.2.0.0)
apache-libcloud (1.5.0)
appdirs (1.4.3)
asn1crypto (0.22.0)
azure (2.0.0rc5)
azure-batch (0.30.0rc5)
azure-common (1.1.4)
azure-graphrbac (0.30.0rc5)
azure-mgmt (0.30.0rc5)
azure-mgmt-authorization (0.30.0rc5)
azure-mgmt-batch (0.30.0rc5)
azure-mgmt-cdn (0.30.0rc5)
azure-mgmt-cognitiveservices (0.30.0rc5)
azure-mgmt-commerce (0.30.0rc5)
azure-mgmt-compute (0.30.0rc5)
azure-mgmt-keyvault (0.30.0rc5)
azure-mgmt-logic (0.30.0rc5)
azure-mgmt-network (0.30.0rc5)
azure-mgmt-notificationhubs (0.30.0rc5)
azure-mgmt-nspkg (2.0.0)
azure-mgmt-powerbiembedded (0.30.0rc5)
azure-mgmt-redis (0.30.0rc5)
azure-mgmt-resource (0.30.0rc5)
azure-mgmt-scheduler (0.30.0rc5)
azure-mgmt-storage (0.30.0rc5)
azure-mgmt-web (0.30.0rc5)
azure-nspkg (2.0.0)
azure-servicebus (0.20.2)
azure-servicemanagement-legacy (0.20.3)
azure-storage (0.32.0)
boto (2.46.1)
boto3 (1.4.4)
botocore (1.5.39)
certifi (2017.1.23)
cffi (1.10.0)
chardet (3.0.2)
cryptography (1.8.1)
docutils (0.13.1)
dopy (0.3.5)
enum34 (1.1.6)
futures (3.1.1)
idna (2.5)
ipaddress (1.0.18)
isodate (0.5.4)
Jinja2 (2.8)
jmespath (0.9.2)
keyring (10.3.2)
MarkupSafe (1.0)
msrest (0.4.1)
msrestazure (0.4.7)
oauthlib (2.0.2)
packaging (16.8)
paramiko (2.1.2)
pip (9.0.1)
pkg-resources (0.0.0)
pyasn1 (0.2.3)
pycparser (2.17)
pycrypto (2.6.1)
PyJWT (1.4.2)
pyOpenSSL (16.2.0)
pyparsing (2.2.0)
python-dateutil (2.6.0)
PyYAML (3.12)
requests (2.13.0)
requests-oauthlib (0.8.0)
s3transfer (0.1.10)
SecretStorage (2.3.1)
setuptools (34.4.1)
six (1.10.0)
wheel (0.30.0a0)

[dnapier]

What are the permissions on the .pem file you use to ssh to your machine?

[Tupoun]

Permissions of the .pem file are 0600. I have only algo.pem file on ~/.ssh. I tried command ssh ubuntu@xx.xx.xx.xx -i ~/.ssh/algo.pem with the same error message Permission denied (publickey).

[Tupoun]

I tried to Install Algo to Digital Ocean cloud and it went through without problems. The error is on EC2 cloud only.

[mister2d]

What are the permissions of your public key on the EC2 cloud instance?

FYI, you might want to not put your public ip address out there.

[dnapier]

Tupoun, I'd give 400 a shot.

edit: This should not be the issue, here's a link to AWS's troubeshooting of this error.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html
Find the section titled: Error: Host key not found, Permission denied (publickey), or Authentication failed, permission denied

[aboutte]

FWIT I have seen this several times (definitely intermittent). A redeploy has always worked for me.

Is the message possibly misleading?

fatal: [35.156.181.105]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '35.156.181.105' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}

It says Permission denied (publickey) and also "unreachable": true. Is unreachable the real issue?

[mister2d]

@aboutte

The message is complaining that it can't read the public key on the remote system because the permissions don't allow it. Using 'chmod 600' or 400 will work if applied to all files within the ".ssh" directory, assuming the user:group permissions are correct.

If the permissions are there, then make sure that the ".ssh" directory is set with 700. "chmod 700 .ssh"

[Tupoun]

I check the permissions of the ".ssh" directory is set to 700, the permissions of the .pem file is 600. If I change it to 400 error is the same (file permissions).

So I decide to start the installation from the scratch. I found one interesting thing. I'm not able to install algo to EC2 in case I choose server eu-central-1 (Frankfurt). In this case I always stop with the error message describe above.

I assume the new instance is add to EC2 before the error message occure (as I see from the log). But the instance is not add to EC2 and this should be way the server return the error message "Failed to connect....."

But if I choose the default server us-east-1 US East (N. Virginia) everything is installed properly.

It seems the problem is with the different server then default one. I use AWS free tire. Could you please someone confirm it?