Failure installing upgrade_package_nonvulnerable_to_vulnerable windows agent test case in E2E VD tests

[Rebits]

Description

It has been detected in Release 4.8.0 - RC 1 - Vulnerability Detection E2E tests, that the node package for upgrade_package_nonvulnerable_to_vulnerable test case fails to be installed in the environment:

2024-05-07 11:11:32 Error installing the package on agent2: Failed to install the package in agent2: {'changed': False, 'msg': 'Unhandled exception while executing module: Exception calling "OpenPackage" with "2" argument(s): "This installation package could not be opened. Verify that the package exists and that you can access it, or contact the application vendor to verify that this is a valid Windows Installer package"'} (remote_operations_handler.py:401)

This fails seems to be erratic and only affects to the Windows agent. Further research is required.

[Rebits]

Manual installation was performed in the Windows agent. Node package was installed without any error:

It is required to research under which conditions reported error is produced

---

After investigating the matter, it appears that the issue may stem from a potentially corrupted node package downloaded during testing. To address this, it's prudent to incorporate some additional validation steps within the installation process.

[Rebits]

It tested the installation of the Windows package using framework. More than 30 times the node package was installed, upgraded, and removed but errors were not replicated

Force windows package install script

from wazuh_testing.tools.system import HostManager

hm = HostManager("inv.yaml")

for i in range(30):
    print("Installing node 18")
    hm.install_package("agent2", "https://nodejs.org/dist/v18.20.0/node-v18.20.0-x64.msi",
                       "windows")
    print("Installing node 20")
    hm.install_package("agent2", "https://nodejs.org/dist/v20.5.1/node-v20.5.1-x64.msi",
                       "windows")
    print("Removing node")
    hm.remove_package("agent2", "windows", "{5674EEF9-AA04-4DEC-9191-67D745D090A2}")

[Rebits]

Proposed fix: d38ffe4

It has included a simple retry logic for certain error events in the environment for package installation.

[Rebits]

Currently testing fix in tests environment

[Rebits]

To evaluate the new development, it's been suggested that we conduct some fundamental tests

import pytest
import testinfra

import wazuh_testing.tools.system


hm = wazuh_testing.tools.system.HostManager("INV.yaml")

def test_install_package():
    hm.install_package("agent3", "https://nodejs.org/dist/v18.20.0/node-v18.20.0-x64.msi", "windows")
    hm.install_package("agent1", "https://dl.grafana.com/oss/release/grafana-8.5.5-1.x86_64.rpm", "centos")
    hm.install_package("agent2", "https://dl.grafana.com/oss/release/grafana_8.5.6_amd64.deb", "ubuntu")


def test_remove_package():
    hm.remove_package("agent1", "centos", "grafana*")
    hm.remove_package("agent2", "ubuntu", "grafana*")
    hm.remove_package("agent3", "windows", "{ED801E31-1556-48A1-AC38-BADEF42169B0}")

def test_install_package_fail_retry():
    with pytest.raises(Exception):
        hm.install_package("agent3", "C:/Users/qa/YourCorruptedFile.msi", "windows")

Result

(qa) ➜  TestingScripts python3 -m pytest -s test_install.py --log-cli-level=debug
==================================== test session starts =====================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.4.0
rootdir: /home/rebits/Wazuh/TestingScripts
plugins: html-3.1.1, testinfra-5.0.0, mock-3.14.0, metadata-3.1.1
collected 3 items                                                                            

test_install.py::test_install_package 
--------------------------------------- live log call ----------------------------------------
DEBUG    root:system.py:557 Package installation result {'changed': True, 'elapsed': 0.35239149999999997, 'msg': 'OK', 'rc': 0, 'reboot_required': False, 'status_code': 200}
DEBUG    root:system.py:557 Package installation result {'ansible_facts': {'pkg_mgr': 'yum'}, 'changed': True, 'changes': {'installed': ['/home/qa/.ansible/tmp/ansible-tmp-1715354609.5784547-66406-129862447645934/grafana-8.5.5-1.x86_64WGh2cc.rpm']}, 'msg': '', 'rc': 0, 'results': ['Loaded plugins: fastestmirror\nExamining /home/qa/.ansible/tmp/ansible-tmp-1715354609.5784547-66406-129862447645934/grafana-8.5.5-1.x86_64WGh2cc.rpm: grafana-8.5.5-1.x86_64\nMarking /home/qa/.ansible/tmp/ansible-tmp-1715354609.5784547-66406-129862447645934/grafana-8.5.5-1.x86_64WGh2cc.rpm to be installed\nResolving Dependencies\n--> Running transaction check\n---> Package grafana.x86_64 0:8.5.5-1 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package      Arch        Version      Repository                          Size\n================================================================================\nInstalling:\n grafana      x86_64      8.5.5-1      /grafana-8.5.5-1.x86_64WGh2cc      250 M\n\nTransaction Summary\n================================================================================\nInstall  1 Package\n\nTotal size: 250 M\nInstalled size: 250 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Installing : grafana-8.5.5-1.x86_64                                       1/1 \n### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd\n sudo /bin/systemctl daemon-reload\n sudo /bin/systemctl enable grafana-server.service\n### You can start grafana-server by executing\n sudo /bin/systemctl start grafana-server.service\nPOSTTRANS: Running script\n  Verifying  : grafana-8.5.5-1.x86_64                                       1/1 \n\nInstalled:\n  grafana.x86_64 0:8.5.5-1                                                      \n\nComplete!\n']}
DEBUG    root:system.py:557 Package installation result {'changed': True, 'stderr': '', 'stderr_lines': [], 'stdout': 'Selecting previously unselected package grafana.\n(Reading database ... 120533 files and directories currently installed.)\nPreparing to unpack .../grafana_8.5.6_amd64il527gz1.deb ...\nUnpacking grafana (8.5.6) ...\nSetting up grafana (8.5.6) ...\nRestarting grafana-server service... OK\n', 'stdout_lines': ['Selecting previously unselected package grafana.', '(Reading database ... 120533 files and directories currently installed.)', 'Preparing to unpack .../grafana_8.5.6_amd64il527gz1.deb ...', 'Unpacking grafana (8.5.6) ...', 'Setting up grafana (8.5.6) ...', 'Restarting grafana-server service... OK']}
PASSED
test_install.py::test_remove_package 
--------------------------------------- live log call ----------------------------------------
INFO     root:system.py:666 Removing package grafana* from host agent1
INFO     root:system.py:667 System: centos
DEBUG    root:system.py:699 Package removed result {'ansible_facts': {'pkg_mgr': 'yum'}, 'changed': True, 'changes': {'removed': ['grafana*']}, 'msg': '', 'rc': 0, 'results': ['Loaded plugins: fastestmirror\nResolving Dependencies\n--> Running transaction check\n---> Package grafana.x86_64 0:8.5.5-1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package     Arch       Version        Repository                          Size\n================================================================================\nRemoving:\n grafana     x86_64     8.5.5-1        @/grafana-8.5.5-1.x86_64WGh2cc     250 M\n\nTransaction Summary\n================================================================================\nRemove  1 Package\n\nInstalled size: 250 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Erasing    : grafana-8.5.5-1.x86_64                                       1/1 \n  Verifying  : grafana-8.5.5-1.x86_64                                       1/1 \n\nRemoved:\n  grafana.x86_64 0:8.5.5-1                                                      \n\nComplete!\n']}
INFO     root:system.py:666 Removing package grafana* from host agent2
INFO     root:system.py:667 System: ubuntu
DEBUG    root:system.py:699 Package removed result {'changed': True, 'stderr': '', 'stderr_lines': [], 'stdout': "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n  libpython2-dev libpython2.7 libpython2.7-dev linux-image-5.13.0-1021-aws\n  linux-modules-5.13.0-1021-aws python2-dev python2.7-dev\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n  grafana\n0 upgraded, 0 newly installed, 1 to remove and 332 not upgraded.\nAfter this operation, 262 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 127246 files and directories currently installed.)\r\nRemoving grafana (8.5.6) ...\r\nNEEDRESTART-VER: 3.5\nNEEDRESTART-KCUR: 5.15.0-1015-aws\nNEEDRESTART-KEXP: 5.15.0-1015-aws\nNEEDRESTART-KSTA: 1\nNEEDRESTART-SVC: grafana-server.service\n", 'stdout_lines': ['Reading package lists...', 'Building dependency tree...', 'Reading state information...', 'The following packages were automatically installed and are no longer required:', '  libpython2-dev libpython2.7 libpython2.7-dev linux-image-5.13.0-1021-aws', '  linux-modules-5.13.0-1021-aws python2-dev python2.7-dev', "Use 'sudo apt autoremove' to remove them.", 'The following packages will be REMOVED:', '  grafana', '0 upgraded, 0 newly installed, 1 to remove and 332 not upgraded.', 'After this operation, 262 MB disk space will be freed.', '(Reading database ... ', '(Reading database ... 5%', '(Reading database ... 10%', '(Reading database ... 15%', '(Reading database ... 20%', '(Reading database ... 25%', '(Reading database ... 30%', '(Reading database ... 35%', '(Reading database ... 40%', '(Reading database ... 45%', '(Reading database ... 50%', '(Reading database ... 55%', '(Reading database ... 60%', '(Reading database ... 65%', '(Reading database ... 70%', '(Reading database ... 75%', '(Reading database ... 80%', '(Reading database ... 85%', '(Reading database ... 90%', '(Reading database ... 95%', '(Reading database ... 100%', '(Reading database ... 127246 files and directories currently installed.)', 'Removing grafana (8.5.6) ...', 'NEEDRESTART-VER: 3.5', 'NEEDRESTART-KCUR: 5.15.0-1015-aws', 'NEEDRESTART-KEXP: 5.15.0-1015-aws', 'NEEDRESTART-KSTA: 1', 'NEEDRESTART-SVC: grafana-server.service']}
INFO     root:system.py:666 Removing package {ED801E31-1556-48A1-AC38-BADEF42169B0} from host agent3
INFO     root:system.py:667 System: windows
DEBUG    root:system.py:699 Package removed result {'changed': True, 'rc': 0, 'reboot_required': False}
PASSED
test_install.py::test_install_package_fail_retry 
--------------------------------------- live log call ----------------------------------------
DEBUG    root:system.py:557 Package installation result {'changed': False, 'msg': 'Unhandled exception while executing module: Exception calling "OpenPackage" with "2" argument(s): "This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer package"'}
ERROR    root:system.py:566 Error installing C:/Users/qa/YourCorruptedFile.msi in agent3:Corrupted package detected. Retrying installation...
DEBUG    root:system.py:557 Package installation result {'changed': False, 'msg': 'Unhandled exception while executing module: Exception calling "OpenPackage" with "2" argument(s): "This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer package"'}
ERROR    root:system.py:566 Error installing C:/Users/qa/YourCorruptedFile.msi in agent3:Corrupted package detected. Retrying installation...
DEBUG    root:system.py:557 Package installation result {'changed': False, 'msg': 'Unhandled exception while executing module: Exception calling "OpenPackage" with "2" argument(s): "This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer package"'}
ERROR    root:system.py:566 Error installing C:/Users/qa/YourCorruptedFile.msi in agent3:Corrupted package detected. Retrying installation...
PASSED

=============================== 3 passed in 137.71s (0:02:17) ================================

Note

These tests shouldn't be uploaded to the repository because they require specific environmental conditions and may not be universally applicable. However, in future functional testing, we should design tests to ensure the framework's consistency, including mocking system results

[rafabailon]

LGTM