DevOps Deployments Metrics

An application that generates DevOps deployment metrics from GitHub repositories using a GitHub Action workflow to deploy a product.

Three deployment metrics

This application reads historical data for workflows from GitHub. The application looks at records for the deployment workflow and generates Deployment Frequency, Change Fail Percentage and a Mean Time to Recover metrics for fixed-width reporting periods.

Metrics will be reported from a configured starting time to the present, with results reported in fixed-width periods. The reporting period width is configurable from 1 day on up.

Deployment Frequency

Deployment Frequency is a simple calculation of the number of successful deployments per day in the reporting period. For example, if there is a 14-day reporting period, and there are 28 successful deployments by the workflow during that period, the deployment frequency would be 2.0.

Change Fail Rate

Change Fail Rate is a simple ratio of the number of failed deployments over the total number of deployments in a given reporting period, expressed as a percentage. If all the deployments fail, the change fail rate will be 100.0. If all the deployments succeed, the change fail rate will be 0.0.

Mean Time to Recover

The deployment Mean Time to Recover (MTTR) mean of the deployment time-to-recover in a given reporting period. The deployment time-to-recover is the time between a failed deployment and the next successful deployment. The metric is reported in hours.

Features

• Generates CSV files with deployment frequency, deployment change-fail percentage and deployment mean time to recover metrics.
• Generates CSV file with deployment log record

Requirements

• Python 3.9+
• Poetry (installation instructions provided here)
• Supported OS: Windows 10, Windows 11, Linux, MacOS

Installation

poetry install

Usage

Create a configuration file

A configuration file must be created and passed to the application at run time.

Sample file: my-config.toml

    # Keys
    title = "Sample devops-deployment-metrics configuration"

    [general]
        time-slice-days = 7
        start-date = 2023-09-01T00:00:01
        date-format = "%Y-%m-%d"
        timezone = "UTC"

    [[repositories]]
        owner = "ccsq-isfcs"
        repo = "security-findings-lambda"
        id = "31952979"
        deployment-frequency = "df"
        change-fail-rate = "cfr"
        mean-time-to-recover = "mttr"
        deployment-log = "deployments"

Field Description

time-slice-days: duration used to group metric data start-date: the start date of metric collection (end date is automatically assumed to be current day) owner: Github organization name repo: repository name **id:**GitHub workflow ID corresponding to the GitHub action that represents CI/CD

Identifying workflow ID

1. Generate a GitHub Personal Access token with rights to read all repositories and workflows in the organization. Instructions are provided here
2. Make a call to the GitHub REST services to list the workflows available in a target repository.

Format:

curl --location 'https://api.github.com/repos/<GITHUB ORGANIZATION NAME>/<REPOSITORY NAME>/actions/workflows' \
--header 'Authorization: Bearer <YOUR PERSONAL ACCESS TOKEN>'

Sample:

curl --location 'https://api.github.com/repos/ccsq-isfcs/security-findings-lambda/actions/workflows' \
--header 'Authorization: Bearer <YOUR PERSONAL ACCESS TOKEN>'

Launching the application

poetry run devops-deployment-metrics -v -c my-config.toml

Please see the Command-line Reference for more details.

Contributing

Contributions are very welcome. To learn more, see the Contributor Guide.

License

Distributed under the terms of the [MIT license], DevOps Deployments Metrics is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Credits

This project was generated from @cjolowicz's Hypermodern Python Cookiecutter template.