Generic Testing Farm tests

This repository contains generic tests for Testing Farm defined in TMT/FMF format.

To run these tests on your localhost:

• install tmt

dnf -y install dnf-plugins-core
dnf -y copr enable psss/tmt
dnf -y install tmt-container tmt-testcloud

• clone this repository:

git clone https://gitlab.com/testing-farm/generic-tests
cd generic-tests

rpmdeplint

To run rpmdeplint for a build, you will need its task ID and release specifiction. The test uses rpmdeplint container from https://pagure.io/fedora-ci/rpmdeplint available via quay.io repository: https://quay.io/repository/testing-farm/test-rpmdeplint

Plan definition: rpmdeplint.fmf

tmt run -a execute --script "python3 /tmp/test_rpmdeplint.py -t 41970645 -r f30" plan --name /rpmdeplint
/var/tmp/tmt/run-192

/rpmdeplint
    discover
        how: shell
        tests: 1 test selected
    provision
        image: quay.io/testing-farm/test-rpmdeplint (force pull)
    prepare
    execute
        how: shell
        args: nohup /var/tmp/tmt/run-192/rpmdeplint/execute/run.sh -v /var/tmp/tmt/run-192/rpmdeplint shell /var/tmp/tmt/run-192/rpmdeplint/execute/stdout.log /var/tmp/tmt/run-192/rpmdeplint/execute/stderr.log
    stdout.log: .D

    stderr.log:
         /var/tmp/tmt/run-192/rpmdeplint $ main shell < discover/tests.yaml
         > /script-00:
         >     path: .
         >     test: python3 /tmp/test_rpmdeplint.py -t 41970645 -r f30
         > > shell execute: cd '/var/tmp/tmt/run-192/rpmdeplint/discover' && python3 /tmp/test_rpmdeplint.py -t 41970645 -r f30
         > > out.log:
        /usr/lib/python3.7/site-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
          InsecureRequestWarning,
        INFO: Running: 'rm -rf *.rpm'...
        INFO: Dowloading packages for build: beaker-27.2-1.fc30...
        INFO: Running: 'koji download-build --arch x86_64 --arch noarch --arch i686 --noprogress --debuginfo --task-id 41970645'...
        Downloading: beaker-client-27.2-1.fc30.noarch.rpm
        Downloading: beaker-common-27.2-1.fc30.noarch.rpm
        INFO: Running: 'set -o pipefail; rpmdeplint check-sat --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/x86_64  --arch=x86_64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee x86_64-check-sat.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-repoclosure --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/x86_64  --arch=x86_64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee x86_64-check-repoclosure.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-conflicts --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/x86_64  --arch=x86_64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee x86_64-check-conflicts.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-upgrade --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/x86_64  --arch=x86_64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee x86_64-check-upgrade.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/x86_64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'rm -rf *.rpm'...
        INFO: Dowloading packages for build: beaker-27.2-1.fc30...
        INFO: Running: 'koji download-build --arch aarch64 --arch noarch --noprogress --debuginfo --task-id 41970645'...
        Downloading: beaker-client-27.2-1.fc30.noarch.rpm
        Downloading: beaker-common-27.2-1.fc30.noarch.rpm
        INFO: Running: 'set -o pipefail; rpmdeplint check-sat --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/aarch64  --arch=aarch64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee aarch64-check-sat.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-repoclosure --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/aarch64  --arch=aarch64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee aarch64-check-repoclosure.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-conflicts --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/aarch64  --arch=aarch64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee aarch64-check-conflicts.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-upgrade --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/aarch64  --arch=aarch64 beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee aarch64-check-upgrade.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/aarch64/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'rm -rf *.rpm'...
        INFO: Dowloading packages for build: beaker-27.2-1.fc30...
        INFO: Running: 'koji download-build --arch armhfp --arch noarch --noprogress --debuginfo --task-id 41970645'...
        Downloading: beaker-client-27.2-1.fc30.noarch.rpm
        Downloading: beaker-common-27.2-1.fc30.noarch.rpm
        INFO: Running: 'set -o pipefail; rpmdeplint check-sat --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/armhfp  --arch=armhfp beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee armhfp-check-sat.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-repoclosure --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/armhfp  --arch=armhfp beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee armhfp-check-repoclosure.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-conflicts --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/armhfp  --arch=armhfp beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee armhfp-check-conflicts.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'set -o pipefail; rpmdeplint check-upgrade --repo Fedora-30-repo,http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os --repo koji-f30-repo,https://kojipkgs.fedoraproject.org/repos/f30-build/latest/armhfp  --arch=armhfp beaker-client-27.2-1.fc30.noarch.rpm beaker-common-27.2-1.fc30.noarch.rpm |& tee armhfp-check-upgrade.log'...
        Failed to download repodata for Repo(repo_name='Fedora-30-repo', baseurl='http://download.fedoraproject.org/pub/fedora/linux/releases/f30/Everything/armhfp/os'): Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
        INFO: Running: 'rm -rf *.rpm'...
         > > exitcode.log:
        0
        result: 1 test passed, 0 tests failed
    report
    finish

Result logs should be available in the run directory, e.g. /var/tmp/tmt/run-192 in the above example.

installability

To run installability for a build, you will need its task ID, release specification and URL to compose.

Plan definition: installability.fmf

Note	make sure this tmt PR is merged: teemtee/tmt#148 (or use it for running)

tmt run -ae TASK_ID=27095830 -e RELEASE=rhel-8.2.0 -d provision -h libvirt.testcloud -i URL_TO_COMPOSE plan --name /installability

rpminspect

To run for a build for a build, you will need its task ID. The test uses fedora container.

Plan definition: rpminspect.fmf

tmt run -ae BUILD=perl-Data-Validate-IP-0.27-11.el8 provision -h podman -i fedora plan --name /rpminspect