A minimal template for Python development

Why?

This repository exists to show a working example of Python formatting, linting and import sorting configurations with continuous integration.

Structure

This is a very simple, single Python module. It is not a Python package and does not contain the structure or tooling required for packaging / installing a Python project.

Formatting

Formatting is handled by black.

Black is an uncompromising Python code formatting tool. It takes a Python file as an input, and provides a reformatted Python file as an output, using rules that are a strict subset of PEP 8. It offers very little in the way of configuration (line length being the main exception) in order to achieve formatting consistency. It is deterministic - it will always produce the same output from the same inputs.

The line length configuration is stored in pyproject.toml.

Configuring Black to automatically run every time code is modified allows developers to forget about how code should be formatted and stay focused on functionality. Many IDEs and text editors allow this. Git hooks can also be used to execute Black before Python code is committed to a repository - this is not configured here as use of autosave and continuous integration checks makes it redundant.

VSCode Configuration

In VSCode, you can set the default Python formatter using the setting: "python.formatting.provider": "black", and then turn on autosave using "editor.formatOnSave": true. VSCode will pick up the line length configuration from a pyproject.toml file per repository.

Linting

Linting is handled by pylint.

Pylint checks Python files in order to detect syntax errors and potential bugs (unreachable code / unused variables), provide refactoring help,

The configuration is stored in pyproject.toml.

VSCode Configuration

Pylint is enabled by default in VSCode. VSCode will respect Pylint configurations stored in a pyproject.toml file per repository.

Sorting Imports

Import sorting is handled by isort.

isort sorts Python imports alphabetically within their respective sections:

1. Standard library imports
2. Related third party imports
3. Local application / library specific imports

isort has many configuration options but these can cause inconsistencies with Black, so must be carefully assessed. The configurations within this repository will provide consistently ordered / formatted imports.

The configuration is stored in pyproject.toml.

VSCode Configuration

It is possible to configure VSCode to run isort every time you save, using the codeActionsOnSave": { "source.organizeImports": true } setting. This currently causes a race condition when formatting by black is also configured to occur on save ("editor.formatOnSave": true), leading to lines being erroneously deleted. The VSCode Python team can't do anything about this.

The configuration stored here works with black - that is to say, black will not reformat changes made by isort - but it is still unsafe to apply both commands simultaneously.

In VSCode, you can manually trigger import sorting by:

• right clicking anywhere within a file and selecting Sort Imports.
• command palette (Ctrl + Shift + P), then Python Refactor: Sort Imports.
• creating a shortcut key for python.sortImports

There is also some discussion over whether black should just handle import sorting itself.

Line Length

Line length needs to be consistently configured for formatting, linting and while sorting imports. In order to change the line length for a specific project, it will need to be updated in pyproject.toml.

Commit Messages

Commit messages must conform to conventional commits:

type(optional-scope): description

where type is one of build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test as specified in .gitlint.

For example:

ci(github-actions): add new matrix jobs for Python 3.8

Continuous Integration

GitHub Actions

The GitHub Actions workflow is stored at .github/workflows/push.yml.

This configuration contains two jobs, with each containing a build matrix with two different versions of Python. The second job depends on the first job being completed successfully by using the needs key within the second job.

Linter

The first job, linter, tests formatting, linting and correct sorting of imports.

gitlint

This command runs gitlint on the most recent commit, to check that commit message titles conform to the conventional commits specification. Earlier commits may not conform, but all commits that are included in a Pull Request are checked by the next command. Pull Requests are the recommended code review workflow before merging into master.

gitlint --commits origin/${{ github.base_ref }}..${{ github.event.pull_request.head.sha }}

This command runs gitlint over all commits in a Pull Request, to check that commit message titles conform to the conventional commits specification.

black . --check --diff

This command runs the black formatter over the Python files in the current directory. The --check flag ensures that the command will error out if any changes are required, rather than making changes. --diff will show the difference.

black will respect the line length configuration stored in pyproject.toml.

pylint salutation.py test_salutation.py

This command runs the pylint checks for errors, bugs, non-conforming code style and potential code structure improvements over our Python module and tests.

pylint uses the configurations stored in pyproject.toml.

isort . --check --diff

This command runs the isort import sorter recursively over the Python files in the current directory. The --check flag ensures that the command will error out if any changes are required, rather than making changes. --diff will show the difference.

isort will respect the configurations stored in pyproject.toml.

Test

The second job, test, runs tests using pytest and verifies test coverage.

pytest

Simply execute pytest and it will find things that look like tests and run them.

Tests run in a random order thanks to pytest-randomly. This helps to detect interdependencies between tests. A test sequence can be reproduced by using the same --randomly-seed=VALUE option. pytest will always print this value, for example:

Using --randomly-seed=1548714831

This randomness can also be used to generate fuzzing input.

Automated Pull Request Merging

Kodiak

Kodiak is enabled on this repository, configured by .kodiak.toml.

Kodiak is a GitHub Bot that automatically merges pull requests that have been labelled automerge :rocket: once CI and required approvals have passed. Kodiak is free for open source repositories, but requires a per user subscription for private repositories.

Automated Dependency Updates

Dependabot

Dependabot is enabled on this repository, configured by .github/dependabot.yml.

Dependabot checks the dependencies listed in requirements-dev.txt daily to check if any new versions of those dependencies have been released. If Dependabot finds a new release, it opens a new Pull Request with a version bump, and attempts to show the changelog for that change. It also provides a compatibility prediction based on all of the Pull Requests that it has opened on other open source repositories and whether continuous integration was successful.

Dependabot is owned by GitHub and free for both open source and private repositories.

Automated Vulnerability Checks

Development

Prerequisites:

• Python 3.8 through 3.12 (this can be changed in .python-version and pyproject.toml)
• Poetry

Optional dependencies:

• Pyenv to use the reference Python version in pyproject.toml with a simple pyenv install

1. Install the project dependencies: poetry install
2. Activate the virtualenv: poetry shell

Install Git hooks:

pre-commit install --hook-type=commit-msg --hook-type=pre-commit --overwrite

They run on every commit to check the repository contents and commit message.