step1 installation fails with "Could not find the requested service docker: host"

[MrPeWu]

Expected Behavior

successful step1

Actual Behavior

step1 fails with:
failed: [10.22.11.116] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
for all nodes

Steps to Reproduce Behavior

I am installing current ECS CE in VMware VMs with centos7 minimal OS using Standard Installation method:

1. Install Centos7 minimal on 4 nodes (install + 3 x ECS node), configure IP addresses and hostnamee, create user admin
2. Clone ECS CE from Git to install node
3. configure deploy.yml
4. bootstrap install node: ./bootstrap.sh -g -c deploy.yml
5. run step1

Relevant Output and Logs

TASK [CentOS_7_baseline_install : CentOS 7 | Start and enable services] ***************************************************************************************************************************************************************************************
failed: [10.22.11.116] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.115] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.117] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.116] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
failed: [10.22.11.115] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
ok: [10.22.11.116] => (item=firewall)
failed: [10.22.11.117] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
ok: [10.22.11.117] => (item=firewall)
ok: [10.22.11.115] => (item=firewall)

---

Notifies: @nikhil-vr

[MrPeWu]

Found ansible/ansible#36585
Fixed with: systemctl daemon-reload

Unfortunately there is a different issue down the road:

ecs-install> Init bittorrent ffx .. OK

PLAY [Common | Ping data nodes before doing anything else] ****************************************************************************************************************************************************************************************************
 [WARNING]: error in 'jsonfile' cache plugin while trying to read /var/cache/emc/ecs-install/facts/10.22.11.115 : Expecting value: line 657 column 21 (char 22319). Most likely a corrupt file, so erasing and failing.

ERROR! Error while decoding the cache file /var/cache/emc/ecs-install/facts/10.22.11.115: The cache file /var/cache/emc/ecs-install/facts/10.22.11.115 was corrupt, or did not otherwise contain valid data. It has been removed, so you can re-run your command now.
Operation failed.

The json file looks perfectly OK.

[nikhil-vr]

Please share your env details and steps tried , so can try to re-produce and look at the issue.

[MrPeWu]

Tried again step by step.
Failed again with:

TASK [CentOS_7_baseline_install : CentOS 7 | Start and enable services] ***************************************************************************************************************************************************************************************
failed: [10.22.11.115] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.116] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.117] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.115] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
failed: [10.22.11.116] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
failed: [10.22.11.117] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}

VM/OS details

4 VMs (VMware, ESXi 7.0.3), 4 vCPU, 24 GB RAM, 30 + 160 GB disk, single NIC (10.22.11.0/24)
Static IPs: 10.22.11.114 (install node), 115-117 (node1-3)
CentOS 7 Minimal installation + yum update = CentOS Linux release 7.9.2009 (Core)

user

admin with default password from deploy.yml

[root@ecsinst ~]# useradd admin
[root@ecsinst ~]# usermod -aG whell admin
[root@ecsinst ~]# passwd admin
[root@ecsinst ~]# visudo
# %wheel        ALL=(ALL)       ALL
%wheel  ALL=(ALL)       NOPASSWD: ALL

git clone and deploy.yml

[admin@ecsinst ~]$ sudo yum install -y git
[admin@ecsinst ~]$ git clone https://github.com/EMCECS/ECS-CommunityEdition
[admin@ecsinst ~]$ cd ECS-CommunityEdition/
[admin@ecsinst ECS-CommunityEdition]$ cp docs/design/reference.deploy.yml deploy.yml
[admin@ecsinst ECS-CommunityEdition]$ vi deploy.yml

licensing:
  license_accepted: false
facts:
  install_node: 10.22.11.114
  management_clients:
    - 0.0.0.0/0
  ssh_defaults:
    ssh_username: admin
    ssh_password: ChangeMe
    ansible_become_pass: ChangeMe
    ssh_crypto: rsa
  node_defaults:
    dns_domain: local
    dns_servers:
      - 10.22.22.53
    ntp_servers:
      - 10.22.11.1
    entropy_source: /dev/urandom
  storage_pool_defaults:
    is_cold_storage_enabled: false
    is_protected: false
    description: Default storage pool description
    ecs_block_devices:
      - /dev/sdb
  storage_pools:
    - name: sp1
      members:
        - 10.22.11.115
        - 10.22.11.116
        - 10.22.11.117
      options:
        is_protected: false
        is_cold_storage_enabled: false
        description: My First SP
        ecs_block_devices:
          - /dev/sdb
  virtual_data_center_defaults:
    description: Default virtual data center description
  virtual_data_centers:
    - name: vdc1
      members:
        - sp1
      options:
        description: My First VDC
  replication_group_defaults:
    description: Default replication group description
    enable_rebalancing: true
    allow_all_namespaces: true
    is_full_rep: false
  replication_groups:
    - name: rg1
      members:
        - vdc1
      options:
        description: My First RG
        enable_rebalancing: true
        allow_all_namespaces: true
        is_full_rep: false
  management_user_defaults:
    is_system_admin: false
    is_system_monitor: false
  management_users:
    - username: admin1
      password: ChangeMe
      options:
        is_system_admin: true
    - username: monitor1
      password: ChangeMe
      options:
        is_system_monitor: true
  namespace_defaults:
    is_stale_allowed: false
    is_compliance_enabled: false
  namespaces:
    - name: ns1
      replication_group: rg1
      administrators:
        - root
      options:
        is_stale_allowed: false
        is_compliance_enabled: false
  object_user_defaults:
    swift_groups_list:
      - users
    s3_expiry_time: 2592000
  object_users:
    - username: object_admin1
      namespace: ns1
      options:
        swift_password: ChangeMe
        swift_groups_list:
          - admin
          - users
        s3_secret_key: ChangeMeChangeMeChangeMeChangeMeChangeMe
        s3_expiry_time: 2592000
    - username: object_user1
      namespace: ns1
      options:
        swift_password: ChangeMe
        s3_secret_key: ChangeMeChangeMeChangeMeChangeMeChangeMe
  bucket_defaults:
    namespace: ns1
    replication_group: rg1
    head_type: s3
    filesystem_enabled: False
    stale_allowed: False
    encryption_enabled: False
    owner: object_admin1
  buckets:
    - name: bucket1
      options:
        namespace: ns1
        replication_group: rg1
        owner: object_admin1
        head_type: s3
        filesystem_enabled: False
        stale_allowed: False
        encryption_enabled: False

bootstrap*

[admin@ecsinst ECS-CommunityEdition]$ ./bootstrap.sh -g -c deploy.yml
> All done bootstrapping your install node.
>
> To continue (after reboot if needed):
>     $ cd /home/admin/ECS-CommunityEdition
> If you have a deploy.yml ready to go (and did not use -c flag):
>     $ sudo cp deploy.yml /opt/emc/ecs-install/
> If not, check out the docs/design and examples directory for references.
> Once you have a deploy.yml, you can start the deployment
> by running:
>
> [WITH Internet access]
>     $ step1
>   [Wait for deployment to complete, then run:]
>     $ step2
>
> [WITHOUT Internet access]
>     $ island-step1
>   [Migrate your install node into the isolated environment and run:]
>     $ island-step2
>   [Wait for deployment to complete, then run:]
>     $ island-step3
>
> The system has indicated it wants to reboot.
> Please reboot BEFORE continuing to ensure this node is
> operating with the latest kernel and system libraries.
>
> Would you like to reboot now?
 <y/N> y

reboot

step1

[admin@ecsinst ECS-CommunityEdition]$ step1

PLAY [Common | Ping data nodes before doing anything else] ****************************************************************************************************************************************************************************************************

TASK [ping] ***************************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.114]
ok: [10.22.11.116]
ok: [10.22.11.115]
ok: [10.22.11.117]

PLAY [Installer | Gather facts and slice into OS groups] ******************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.117]
ok: [10.22.11.115]

TASK [group_by] ***********************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.115]
ok: [10.22.11.117]
ok: [10.22.11.116]
ok: [10.22.11.114]

PLAY [CentOS 7 | Configure access] ****************************************************************************************************************************************************************************************************************************

TASK [CentOS_7_configure_ssh : CentOS 7 | Distribute ed25519 ssh key] *****************************************************************************************************************************************************************************************

TASK [CentOS_7_configure_ssh : CentOS 7 | Distribute rsa ssh key] *********************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]
ok: [10.22.11.114]

TASK [CentOS_7_configure_ssh : CentOS 7 | Disable SSH UseDNS] *************************************************************************************************************************************************************************************************
ok: [10.22.11.114]
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_configure_ssh : CentOS 7 | Disable requiretty] *************************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.114]
ok: [10.22.11.115]
ok: [10.22.11.117]

TASK [CentOS_7_configure_ssh : CentOS 7 | Disable sudo password reverification for admin group] ***************************************************************************************************************************************************************
ok: [10.22.11.114]
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_configure_ssh : CentOS 7 | Disable sudo password reverification for wheel group] ***************************************************************************************************************************************************************
ok: [10.22.11.114]
changed: [10.22.11.116]
changed: [10.22.11.115]
changed: [10.22.11.117]

TASK [firewalld_configure_access : Firewalld | Ensure service is started] *************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.114]
ok: [10.22.11.117]
ok: [10.22.11.115]

TASK [firewalld_configure_access : Firewalld | Add install node to firewalld trusted zone] ********************************************************************************************************************************************************************
changed: [10.22.11.114]
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [firewalld_configure_access : Firewalld | Add all data nodes to firewalld trusted zone] ******************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=10.22.11.115)
changed: [10.22.11.114] => (item=10.22.11.115)
changed: [10.22.11.117] => (item=10.22.11.115)
changed: [10.22.11.115] => (item=10.22.11.115)
changed: [10.22.11.114] => (item=10.22.11.116)
changed: [10.22.11.116] => (item=10.22.11.116)
changed: [10.22.11.117] => (item=10.22.11.116)
changed: [10.22.11.115] => (item=10.22.11.116)
changed: [10.22.11.114] => (item=10.22.11.117)
changed: [10.22.11.116] => (item=10.22.11.117)
changed: [10.22.11.117] => (item=10.22.11.117)
changed: [10.22.11.115] => (item=10.22.11.117)

TASK [firewalld_configure_access : Firewalld | Whitelist management prefixes] *********************************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=0.0.0.0/0)
changed: [10.22.11.114] => (item=0.0.0.0/0)
changed: [10.22.11.115] => (item=0.0.0.0/0)
changed: [10.22.11.117] => (item=0.0.0.0/0)

TASK [firewalld_configure_access : Firewalld | Add all public service ports to firewalld public zone] *********************************************************************************************************************************************************
changed: [10.22.11.114] => (item=3218/tcp)
changed: [10.22.11.116] => (item=3218/tcp)
changed: [10.22.11.115] => (item=3218/tcp)
changed: [10.22.11.117] => (item=3218/tcp)
changed: [10.22.11.114] => (item=9020-9025/tcp)
changed: [10.22.11.116] => (item=9020-9025/tcp)
changed: [10.22.11.115] => (item=9020-9025/tcp)
changed: [10.22.11.117] => (item=9020-9025/tcp)
changed: [10.22.11.114] => (item=9040/tcp)
changed: [10.22.11.116] => (item=9040/tcp)
changed: [10.22.11.117] => (item=9040/tcp)
changed: [10.22.11.115] => (item=9040/tcp)

TASK [firewalld_configure_access : Firewalld | Ensure service is started] *************************************************************************************************************************************************************************************
changed: [10.22.11.114]
changed: [10.22.11.116]
changed: [10.22.11.115]
changed: [10.22.11.117]

RUNNING HANDLER [CentOS_7_configure_ssh : CentOS 7 | Restart sshd] ********************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

PLAY [Common | Configure hostnames] ***************************************************************************************************************************************************************************************************************************

TASK [common_set_hostname : include_vars] *********************************************************************************************************************************************************************************************************************
ok: [10.22.11.117]
ok: [10.22.11.116]
ok: [10.22.11.115]

TASK [common_set_hostname : Common | Find node hostname] ******************************************************************************************************************************************************************************************************
ok: [10.22.11.115] => (item=(0, u'10.22.11.115'))
ok: [10.22.11.117] => (item=(2, u'10.22.11.117'))
ok: [10.22.11.116] => (item=(1, u'10.22.11.116'))

TASK [common_set_hostname : Common | Set node hostname] *******************************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.115]
changed: [10.22.11.117]

PLAY [Common | Configure /etc/hosts] **************************************************************************************************************************************************************************************************************************

TASK [common_etc_hosts : Common | Add install node to /etc/hosts] *********************************************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=10.22.11.114)
changed: [10.22.11.114] => (item=10.22.11.114)
changed: [10.22.11.117] => (item=10.22.11.114)
changed: [10.22.11.115] => (item=10.22.11.114)

TASK [common_etc_hosts : Common | Add data nodes to /etc/hosts] ***********************************************************************************************************************************************************************************************
changed: [10.22.11.114] => (item=10.22.11.115)
changed: [10.22.11.116] => (item=10.22.11.115)
changed: [10.22.11.115] => (item=10.22.11.115)
changed: [10.22.11.117] => (item=10.22.11.115)
changed: [10.22.11.114] => (item=10.22.11.116)
changed: [10.22.11.116] => (item=10.22.11.116)
changed: [10.22.11.114] => (item=10.22.11.117)
changed: [10.22.11.115] => (item=10.22.11.116)
changed: [10.22.11.117] => (item=10.22.11.116)
changed: [10.22.11.116] => (item=10.22.11.117)
changed: [10.22.11.117] => (item=10.22.11.117)
changed: [10.22.11.115] => (item=10.22.11.117)

PLAY [Common | Test inter-node access] ************************************************************************************************************************************************************************************************************************

TASK [common_access_test : Common | Check node connectivity by IP] ********************************************************************************************************************************************************************************************
ok: [10.22.11.116] => (item=10.22.11.115)
ok: [10.22.11.114] => (item=10.22.11.115)
ok: [10.22.11.116] => (item=10.22.11.116)
ok: [10.22.11.115] => (item=10.22.11.115)
ok: [10.22.11.117] => (item=10.22.11.115)
ok: [10.22.11.114] => (item=10.22.11.116)
ok: [10.22.11.116] => (item=10.22.11.117)
ok: [10.22.11.114] => (item=10.22.11.117)
ok: [10.22.11.117] => (item=10.22.11.116)
ok: [10.22.11.115] => (item=10.22.11.116)
ok: [10.22.11.117] => (item=10.22.11.117)
ok: [10.22.11.115] => (item=10.22.11.117)

TASK [common_access_test : Common | Check node connectivity by short name] ************************************************************************************************************************************************************************************
ok: [10.22.11.114] => (item=luna)
ok: [10.22.11.115] => (item=luna)
ok: [10.22.11.116] => (item=luna)
ok: [10.22.11.117] => (item=luna)
ok: [10.22.11.114] => (item=phobos)
ok: [10.22.11.116] => (item=phobos)
ok: [10.22.11.115] => (item=phobos)
ok: [10.22.11.117] => (item=phobos)
ok: [10.22.11.114] => (item=deimos)
ok: [10.22.11.116] => (item=deimos)
ok: [10.22.11.115] => (item=deimos)
ok: [10.22.11.117] => (item=deimos)

TASK [common_access_test : Common | Check node connectivity by fqdn] ******************************************************************************************************************************************************************************************
ok: [10.22.11.114] => (item=luna)
ok: [10.22.11.116] => (item=luna)
ok: [10.22.11.115] => (item=luna)
ok: [10.22.11.117] => (item=luna)
ok: [10.22.11.114] => (item=phobos)
ok: [10.22.11.116] => (item=phobos)
ok: [10.22.11.114] => (item=deimos)
ok: [10.22.11.117] => (item=phobos)
ok: [10.22.11.115] => (item=phobos)
ok: [10.22.11.116] => (item=deimos)
ok: [10.22.11.117] => (item=deimos)
ok: [10.22.11.115] => (item=deimos)

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************
10.22.11.114               : ok=18   changed=7    unreachable=0    failed=0
10.22.11.115               : ok=23   changed=13   unreachable=0    failed=0
10.22.11.116               : ok=23   changed=13   unreachable=0    failed=0
10.22.11.117               : ok=23   changed=13   unreachable=0    failed=0

Playbook run took 0 days, 0 hours, 0 minutes, 42 seconds


PLAY [Common | Ping data nodes before doing anything else] ****************************************************************************************************************************************************************************************************

TASK [ping] ***************************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.115]
ok: [10.22.11.117]

PLAY [Installer | Slice nodes into OS groups] *****************************************************************************************************************************************************************************************************************

TASK [group_by] ***********************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.115]
ok: [10.22.11.116]
ok: [10.22.11.117]

PLAY [Installer | Perform preflight check] ********************************************************************************************************************************************************************************************************************

TASK [common_collect_facts : Common | Create custom facts directory] ******************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_collect_facts : Common | Insert data_node.fact file] *********************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_collect_facts : Common | Reload facts to pick up new items] **************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.117]
ok: [10.22.11.115]

TASK [common_baseline_check : include_vars] *******************************************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.115]
ok: [10.22.11.117]

TASK [common_baseline_check : Common | Check RAM size] ********************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Check CPU architecture] ************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Validate OS distribution] **********************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | (Optional) Check UTC Timezone] *****************************************************************************************************************************************************************************************
ok: [10.22.11.115] => {
    "msg": "Timezone is not set to UTC - It is recommended to use the UTC timezone"
}
ok: [10.22.11.117] => {
    "msg": "Timezone is not set to UTC - It is recommended to use the UTC timezone"
}
ok: [10.22.11.116] => {
    "msg": "Timezone is not set to UTC - It is recommended to use the UTC timezone"
}

TASK [common_baseline_check : Common | Make sure /data directory does not exist] ******************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.117]
ok: [10.22.11.115]

TASK [common_baseline_check : fail] ***************************************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Make sure /host directory does not exist] ******************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.115]
ok: [10.22.11.117]

TASK [common_baseline_check : fail] ***************************************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Make sure block device(s) exist on node] *******************************************************************************************************************************************************************************
ok: [10.22.11.116] => (item=/dev/sdb)
ok: [10.22.11.115] => (item=/dev/sdb)
ok: [10.22.11.117] => (item=/dev/sdb)

TASK [common_baseline_check : fail] ***************************************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Make sure block device(s) are at least 100GB] **************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Make sure block device(s) are unpartitioned] ***************************************************************************************************************************************************************************
ok: [10.22.11.116] => (item=/dev/sdb)
ok: [10.22.11.115] => (item=/dev/sdb)
ok: [10.22.11.117] => (item=/dev/sdb)

TASK [common_baseline_check : fail] ***************************************************************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Check for listening layer 4 ports] *************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.115]
changed: [10.22.11.117]

TASK [common_baseline_check : Common | Report any conflicts with published ECS ports] *************************************************************************************************************************************************************************

TASK [common_baseline_check : Common | Report any conflicts with internal ECS ports] **************************************************************************************************************************************************************************

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************
10.22.11.115               : ok=12   changed=3    unreachable=0    failed=0
10.22.11.116               : ok=12   changed=3    unreachable=0    failed=0
10.22.11.117               : ok=12   changed=3    unreachable=0    failed=0

Playbook run took 0 days, 0 hours, 0 minutes, 16 seconds


PLAY [Installer | Build the package cache] ********************************************************************************************************************************************************************************************************************

TASK [installer_build_cache : include_vars] *******************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [installer_build_cache : Installer | Create cache directories] *******************************************************************************************************************************************************************************************
changed: [localhost] => (item=/var/cache/emc/ecs-install/os_CentOS_7)
changed: [localhost] => (item=/var/cache/emc/ecs-install/os_CentOS_7)

TASK [installer_build_cache : Installer | Create compressed cache files] **************************************************************************************************************************************************************************************
changed: [localhost -> 10.22.11.114] => (item=/var/cache/emc/ecs-install/os_CentOS_7/package_cache.tgz)
changed: [localhost -> 10.22.11.114] => (item=/var/cache/emc/ecs-install/os_CentOS_7/package_cache.tgz)

TASK [installer_build_cache : Installer | Create cache distribution torrent file] *****************************************************************************************************************************************************************************
changed: [localhost]

PLAY [Installer | Enable torrent ffx] *************************************************************************************************************************************************************************************************************************

TASK [file] ***************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=5    changed=4    unreachable=0    failed=0

Playbook run took 0 days, 0 hours, 3 minutes, 9 seconds

ecs-install> Init bittorrent ffx .. OK

PLAY [Common | Ping data nodes before doing anything else] ****************************************************************************************************************************************************************************************************

TASK [ping] ***************************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.115]
ok: [10.22.11.117]

PLAY [Installer | Slice nodes into OS groups for package install] *********************************************************************************************************************************************************************************************

TASK [group_by] ***********************************************************************************************************************************************************************************************************************************************
ok: [10.22.11.117]
ok: [10.22.11.116]
ok: [10.22.11.115]

PLAY [CentOS 7 | Synchronize cache] ***************************************************************************************************************************************************************************************************************************

TASK [CentOS_7_sync_caches_prep : CentOS 7 | Fetch torrent package names] *************************************************************************************************************************************************************************************
changed: [10.22.11.116 -> 10.22.11.114]
changed: [10.22.11.117 -> 10.22.11.114]
changed: [10.22.11.115 -> 10.22.11.114]

TASK [CentOS_7_sync_caches_prep : CentOS 7 | Make yum cache paths for torrent packages] ***********************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.115] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.117] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.116] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.115] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.117] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)

TASK [CentOS_7_sync_caches_prep : CentOS 7 | Copy torrent packages to nodes] **********************************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.115] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.117] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.116] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.117] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)
changed: [10.22.11.115] => (item=/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm)

TASK [CentOS_7_sync_caches_prep : CentOS 7 | Install torrent packages on nodes] *******************************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=[u'/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm', u'/var/cache/yum/x86_64/7/epel/packages/aria2-1.34.0-5.el7.x86_64.rpm'])
changed: [10.22.11.115] => (item=[u'/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm', u'/var/cache/yum/x86_64/7/epel/packages/aria2-1.34.0-5.el7.x86_64.rpm'])
changed: [10.22.11.117] => (item=[u'/var/cache/yum/x86_64/7/base/packages/c-ares-1.10.0-3.el7.x86_64.rpm', u'/var/cache/yum/x86_64/7/epel/packages/aria2-1.34.0-5.el7.x86_64.rpm'])

TASK [common_sync_caches_by_torrent : include_vars] ***********************************************************************************************************************************************************************************************************
ok: [10.22.11.117]
ok: [10.22.11.116]
ok: [10.22.11.115]

TASK [common_sync_caches_by_torrent : Selinux | Set selinux permissive] ***************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_sync_caches_by_torrent : Common | Create ecs-install host directory on nodes] ********************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_sync_caches_by_torrent : Common | Create ecs-install cache directory on nodes] *******************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_sync_caches_by_torrent : Common | Copy cache torrent digest to nodes] ****************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [common_sync_caches_by_torrent : Common | Torrent sync caches with nodes] ********************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

PLAY [CentOS 7 | Install cached packages] *********************************************************************************************************************************************************************************************************************

TASK [CentOS_7_baseline_install : include_vars] ***************************************************************************************************************************************************************************************************************
ok: [10.22.11.117]
ok: [10.22.11.116]
ok: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Set selinux permissive] ******************************************************************************************************************************************************************************************
ok: [10.22.11.116]
ok: [10.22.11.117]
ok: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Add SSL proxy cert] **********************************************************************************************************************************************************************************************

TASK [CentOS_7_baseline_install : CentOS 7 | Stop and disable unused services] ********************************************************************************************************************************************************************************
changed: [10.22.11.116] => (item=mta)
changed: [10.22.11.115] => (item=mta)
changed: [10.22.11.117] => (item=mta)

TASK [CentOS_7_baseline_install : CentOS 7 | Configure DNS] ***************************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Unpack package_cache.tgz] ****************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Install all packages in the cache] *******************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Configure ntp] ***************************************************************************************************************************************************************************************************
changed: [10.22.11.116]
changed: [10.22.11.117]
changed: [10.22.11.115]

TASK [CentOS_7_baseline_install : CentOS 7 | Start and enable services] ***************************************************************************************************************************************************************************************
failed: [10.22.11.116] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.117] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.115] (item=docker) => {"changed": false, "failed": true, "item": "docker", "msg": "Could not find the requested service docker: host"}
failed: [10.22.11.116] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
failed: [10.22.11.117] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
failed: [10.22.11.115] (item=ntp) => {"changed": false, "failed": true, "item": "ntp", "msg": "Could not find the requested service ntpd: host"}
ok: [10.22.11.116] => (item=firewall)
ok: [10.22.11.117] => (item=firewall)
ok: [10.22.11.115] => (item=firewall)

RUNNING HANDLER [CentOS_7_baseline_install : CentOS 7 | Stop ntp] *********************************************************************************************************************************************************************************************

RUNNING HANDLER [CentOS_7_baseline_install : CentOS 7 | Set the clock] ****************************************************************************************************************************************************************************************

RUNNING HANDLER [CentOS_7_baseline_install : CentOS 7 | Start ntp] ********************************************************************************************************************************************************************************************

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************
10.22.11.115               : ok=19   changed=14   unreachable=0    failed=1
10.22.11.116               : ok=19   changed=14   unreachable=0    failed=1
10.22.11.117               : ok=19   changed=14   unreachable=0    failed=1

Playbook run took 0 days, 0 hours, 3 minutes, 13 seconds
Operation failed.

Last time I was somehow able to get past this step by rebooting all VMs, running sudo systemctl daemon-reload and running step1 again.

It did not help this time..

[MrPeWu]

I finally managed to install ECS.
Several packages had to be installed manually on ECS nodes before running step1:
docker-ce, ntp, pigz, epel-release, python3, docker-py (with pip3 as root)
I have aslo disabled selinux, but I am not sure if that had any effect.

[nikhil-vr]

I have tried to install manually , only package has to install manually was git.