cleanup procedure is not cleaning up

[ghomem]

Describe the bug

The code that should trigger a conf.d/*.conf cleanup is not cleaning up

To reproduce

Steps to reproduce the behavior:

nginx_config_cleanup     : '{{ my_cleanup }}'

    # this clean up only happens if the above variable is true
    nginx_config_cleanup_paths:
      - directory:
          - /etc/nginx/conf.d
        recurse: false
    nginx_config_cleanup_files:
      - /etc/nginx/conf.d/default.conf

TASK [nginxinc.nginx_config : Find NGINX config files] ************************************************************************************************************************************************************************************************************************
ok: [testing-snap05.MYDOMAIN.TLD] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': False}) => {"ansible_loop_var": "item", "changed": false, "examined": 13, "files": [{"atime": 1616512425.5879788, "ctime": 1616512425.0159645, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652034, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616512424.6879563, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLDX.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 508, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1616511920.0872648, "ctime": 1616511919.343246, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652037, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616511918.9712367, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 507, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1616512481.8013923, "ctime": 1616512481.229378, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652039, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616512480.8773692, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLDZ.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 508, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1616512450.800613, "ctime": 1616512450.1845973, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652038, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616512449.8325884, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLDY.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 508, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}], "item": {"directory": ["/etc/nginx/conf.d"], "recurse": false}, "matched": 4, "msg": ""}                                                                                                                                                                                          

TASK [nginxinc.nginx_config : Remove NGINX config files] **********************************************************************************************************************************************************************************************************************
ok: [testing-snap05.MYDOMAIN.TLD] => (item=/etc/nginx/conf.d/default.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/nginx/conf.d/default.conf", "path": "/etc/nginx/conf.d/default.conf", "state": "absent"}

Expected behavior

I expected /etc/nginx/conf.d/*.conf to be deleted before new flies are created. But only default.conf is deleted.

Your environment:

ansible-role-nginx-config from git
nginxinc.nginx, 0.19.1
ansible: 2.9.6+dfsg-1
ubuntu 20.04

[alessfg]

Good catch. The tests didn't account for anything other than default.conf being deleted. The incoming PR should address this bug as well as add a proper test case for a whole directory being cleaned up.

[ghomem]

After git pull, it still does not clean up for me. Only cleans up default.conf. Tried recurse set to true and false.

(item=/etc/nginx/conf.d/default.conf) => {"ansible_loop_var": "item", "changed": false, "item": "/etc/nginx/conf.d/default.conf", "path": "/etc/nginx/conf.d/default.conf", "state": "absent"}

[ghomem]

After Jinja2 update the clean up still does not work. Shall we reopen this?

[ghomem]

TASK [nginxinc.nginx_config : Find NGINX config files] ************************************************************************************************************************************************************************************************************************
ok: [testing-snap05.MYDOMAIN.TLD] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': True}) => {"ansible_loop_var": "item", "changed": false, "examined": 7, "files": [{"atime": 1616759901.1814494, "ctime": 1616759896.8093383, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652060, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616759896.5933328, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD_adhoc.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 2, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, {"atime": 1616777440.7914264, "ctime": 1616777440.3514154, "dev": 64780, "gid": 0, "gr_name": "root", "inode": 652158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1616777440.13941, "nlink": 1, "path": "/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf", "pw_name": "root", "rgrp": true, "roth": true, "rusr": true, "size": 1255, "uid": 0, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}], "item": {"directory": ["/etc/nginx/conf.d"], "recurse": true}, "matched": 2, "msg": ""}                                                                                                                                                                                                                                        

TASK [nginxinc.nginx_config : Remove NGINX config files] **********************************************************************************************************************************************************************************************************************

[alessfg]

The new tests are passing, so I am not entirely sure that this is a bug in the role. I also tried using the same file name as in your example and it got correctly deleted.

Molecule test

TASK [ansible-role-nginx-config : Find NGINX config files] *********************
ok: [ubuntu-bionic] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': False})
ok: [centos-7] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': False})
ok: [debian-buster] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': False})
ok: [alpine-3.10] => (item={'directory': ['/etc/nginx/conf.d'], 'recurse': False})

TASK [ansible-role-nginx-config : Remove NGINX config files] *******************
changed: [centos-7] => (item=/etc/nginx/conf.d/default.conf)
changed: [debian-buster] => (item=/etc/nginx/conf.d/default.conf)
changed: [alpine-3.10] => (item=/etc/nginx/conf.d/default.conf)
changed: [ubuntu-bionic] => (item=/etc/nginx/conf.d/default.conf)
changed: [centos-7] => (item=/etc/nginx/conf.d/mock.conf)
changed: [debian-buster] => (item=/etc/nginx/conf.d/mock.conf)
changed: [alpine-3.10] => (item=/etc/nginx/conf.d/mock.conf)
changed: [ubuntu-bionic] => (item=/etc/nginx/conf.d/mock.conf)
changed: [centos-7] => (item=/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf)
changed: [ubuntu-bionic] => (item=/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf)
changed: [debian-buster] => (item=/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf)
changed: [alpine-3.10] => (item=/etc/nginx/conf.d/testing-snap05.staging.MYDOMAIN.TLD.conf)
ok: [centos-7] => (item=/etc/nginx/conf.d/default.conf)
ok: [ubuntu-bionic] => (item=/etc/nginx/conf.d/default.conf)
ok: [debian-buster] => (item=/etc/nginx/conf.d/default.conf)
ok: [alpine-3.10] => (item=/etc/nginx/conf.d/default.conf)

Have you tried updating to the latest Ansible release? The modules used behind the scenes might have been updated since Ansible 2.9.6. (The absolute minimum Ansible release you should be using these days is 2.9.10 due to its native support of Ansible collections.)

[ghomem]

I have Ansible 2.9.6 which is the latest available for latest Ubuntu LTS, which is Ubuntu 20.04.

[alessfg]

You can install the latest release using pip or Ansible's package repository https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-on-ubuntu. Otherwise you are stuck with the latest release available on Ubuntu's package repository, which is usually not ideal.

[rhombl4]

Got this error on mine too. Latest available ansible from brew:

 > ansible --version
ansible [core 2.11.3]

with debug

  register: nginx_config_files

- debug:
    msg: "{{ nginx_config_files.files  }}"

- name: Remove NGINX config files

I'm getting

TASK [nginxinc.nginx_config : Find NGINX config files] *********************************************************************************
ok: [ansible-test] => (item={'directory': ['/etc/nginx/conf.d', '/etc/nginx/snippets'], 'recurse': True})

TASK [nginxinc.nginx_config : debug] ***************************************************************************************************
fatal: [ansible-test]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'files'\n\nThe error appears to be in '...../roles/nginxinc.nginx_config/tasks/config/cleanup-config.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- debug:\n  ^ here\n"}

[alessfg]

Like I mentioned in an earlier comments, there are explicit tests for this use case now, and they are all passing (https://github.com/nginxinc/ansible-role-nginx-config/blob/main/molecule/cleanup_module/converge.yml). There might be some edge cases that still cause the role to fail, but I'd need more details to be able to reproduce it myself. From what I can tell given your task output, the issue might be that the recurse keyword belongs in each directory dictionary item, not outside those items (check the Molecule test I linked earlier for more details).

P.S.: nginx_config_files.files is not used (https://github.com/nginxinc/ansible-role-nginx-config/blob/main/tasks/config/cleanup-config.yml#L11-L17) 😄

[rhombl4]

@alessfg Yes, I actually just found this. Come here to write about my solution :)
The issue is that the latest release published in Galaxy(which I use for my playbooks) contain 0.3.3 version. These changes you write about are absent at this tag and only present only at main. So I switched source to git repository for requirements.yml.