Skip to content

How to release

Jump to bottom Edit New page
Jens Hedegaard Nielsen edited this page May 3, 2024 · 76 revisions

How to do a release

First, do a prerelease!

Roughly a week's time before the actual planned release date, you should do a prerelease. It amounts to almost the same as doing a release. You write a changelog as described in step 1., and check out the release branch exactly as described in step 2. The difference is that in step 3, you use a prerelease tag (a "release candidate" tag, see PEP 440), which for release v0.N.0 would be v0.N.0rc1.

The release process

Ok, the release process with our new triggering of the pipeline made releasing a lot easier. Below this section you can still find the manual way of releasing.

1. Generate the changelog

We are generating the changelog using Towncrier from the newsfragment submitted with each pull request. Note that due to a logic bug in the config of Towncrier before 22.8.0 you must use at least Towncrier 22.8.0

To generate the changelog for a new release do:

  • Ensure that the preview changelog looks correct.
    1. It should contain all important changes
    2. Render correctly without warnings
    3. All the news fragments are included. If one is missing it likely has a incorrect name. Such as prnumber.Underthehood rather than prnumber.underthehood
  • pip install towncrier
  • create a new git branch for the changelog.
  • towncrier build --version whateverthenewversionis
  • modify docs\changes\index.rst to include the new changelog.
  • Build the docs locally and make sure that everything looks good.
  • OPEN a PR against master. Review it, fix, etc.

You may want to check that all important pull requests are included in the changelog. The command git log <last release>..HEAD is useful to print all changes since last release. You can add --merges --first-parent to the git log command to only show merge commits. --reverse is helpful too. That is: git log <last release>..HEAD --merges --first-parent --reverse Another approach is to get a list of merged pull-request since the last release: find the merged pull-request of the previous release, note the date when that happened, and then paste the following into the "Filter" field in the "Pull requests" tab on GitHub: is:pr is:closed merged:>YYYY-MM-DD -label:dependencies where YYYY-MM-DD should be replaced by the date of the merge of the pull-request of the previous release, and where the -label part excludes PRs with the given label.

2. Make a release branch

Create a separate branch called release/v<MAJOR version number>.<MINOR version number>.x like release/v0.8.x where x is literally x. This branch has to reflect of the QCoDeS version to be released. In most cases this is simply identical to master. Note that creating this branch allows us to keep merging features to master without worry that those will break this release that we are trying to make, and it also allows us to easily make hotfix releases (read below)

3. Github release the package

Make a release on GitHub:

  • from the release/v.n.x branch created above (!), go to Code->Releases->New Draft, copy changelog into the description, give the release a title that is consistent with previous releases, use v<version> as a tag (notice the v; for example, v0.8.0).
  • Save as draft
  • Make sure that CI passes on the release branch. Re-run the pipeline if you're in doubt. The pipelines can be found here
  • Click publish to publish the release

4. Check zenodo DOI

We are using the zenodo link that points automatically to the latest qcodes version on zenodo. Check if badge works as expected and skip this step :-).

Just in case, here are the steps to manually update the zenodo DOI link in the README.rst file:

  • For that update the zenodo doi link in README.rst to the new one (see previous step) via a pull-request.
  • it's important that the main link is via doi.org, not zenodo.org.
  • The easiest way is to go to the webpage of the new "record" on zenodo (for example this), look at the right side of the webpage for a banner image saying DOI | <some URL>, and then click on it - a window will pop up with ways to reference the zenodo "record" page, and from there just take the links from the reStructedText section and paste them to appropriate place in the README.rst

5. Update conda-forge package

QCoDeS is released via conda-forge using a recipe defined in the qcodes-feedstock repository . Upon each new release of QCoDeS a bot should open a pull request against qcodes-feedstock to build the next release. Ensure that pull-request is merged following the instructions within the pull request. If you are currently not a maintainer of the qcodes-feedstock you can add your self to the list of maintainers within the recipe/meta.yaml file and submit a new pull request to be added to the list of maintainers.

6. Finally put an announcement with links to the changelog in Github Discussions, Teams, and elsewhere. If the release has landed on PyPi but not conda-forge (due to the extra steps to land there) please make sure to post; The new version is available on pypi and will be available on conda-forge in a few hours.

Releasing a backport (aka patch version)

To add bugfixes to the latest released version, we create a new release with a name of e.g. v0.8.1 - we increment the last patch number of the release string (as per semantic versioning approach).

  • Create a new change log e.g. 0.8.1 and add it to the index - same as for the normal release
  • We reuse the release/v0.8.x branch and cherry pick the merge commits (git cherry-pick -m 1 <hash>) of the bugfix PRs and the changelog PR onto that branch
  • The remaining steps for releasing via github are just as above

Manual release steps

Below each step is marked with a "check-box icon" and bullet points within each step may mean sub-steps or just useful information, so make sure to read all the bullet points before executing a "check-box" step.

  • Start by making a clean checkout of the latest/current master. Avoid using your local development folder, as this is probably full of unchecked-in files and random junk.

  • cd into that new folder, activate your qcodes python environment, and then do the following.

  • If you are doing a new major or minor release create branch for the release/<version>, for example, release/v0.3.x (note the x) on the main qcodes repo; not on your fork. See https://semver.org/ for definition of release types.

    • If you are doing a new patch release, checkout the branch matching that major minor version and back port the fixes that you want to add to the patch.

  • Update changelogs in docs/changes/<version>.rst (for example, docs/changes/0.3.0.rst).

    • The command git log <lastrelease>..HEAD is useful to print all changes since last release. You can add --merges --first-parent to the git log command to only show merge commits.
    • Another approach is to get a list of merged pull-request since the last release: find the merged pull-request of the previous release, note the date when that happened, and then paste the following into the "Filter" field in the "Pull requests" tab on GitHub: is:pr is:closed merged:>YYYY-MM-DD where YYYY-MM-DD should be replaced by the date of the merge of the pull-request of the previous release.

  • Update index in in docs/changes/index.rst to contain reference to the new <version>.rst file (for example, 0.3.0.rst)

  • OPEN a PR against master. Review it, fix, etc

  • When the PR is done, squash-merge it (into one commit)

  • Create release on GitHub, paste changelog (from that <version>.rst file) into release description. Specify the name of the tag, the format of the tag name should be like v0.3.0.

    • Note that since zenodo.org is connected to our repo, tagging a release will create a new "record" on zenodo. We will later need to take the URL to that new zenodo record, and add it to our README.rst in new PR, but see below about this.

  • Checkout to master locally and pull the tagged master branch from github

  • build locally (make sure you have wheel installed):

    python setup.py sdist bdist_wheel

  • Now let's test the locally-built qcodes.

    • Setup a fresh test environment, e.g. by 
      conda create -n releasetest python=3.7
    • and install the the package from the newly generated wheel/sdist into this environment: 
      pip install dist/qcodes-<version>-py3-none-any.whl
    • Install the additional requirements into the test environment; 
      pip install -r requirements.txt -r test_requirements.txt
    • Now navigate away from the qcodes source checkout and run import qcodes; qcodes.test() from a python shell (within the created test environment) verifying that the tests also pass in this way.

  • upload to pypitest (which is the same as pypi but used for testing registering of python packages):

    • Navigate back to the source folder for the qcodes release and do: 
      twine upload -r pypitest dist/*
    • if the above fails with a 403 authentication error, try first pip install keyring and then adding your user and username with: 
      keyring set https://test.pypi.org/legacy/ <user-name>
      that will prompt you for your password. Then do 
      twine upload --repository-url https://test.pypi.org/legacy/
    • This assumes that you have a ~/.pypirc file with config like below (NOTE: may not work on Windows. The keyring approach described above is probably better on Windows): 
      [distutils]
index-servers =
  pypi
  pypitest

[pypi]
username=your-pypi-username
password=your-pypi-password

[pypitest]
repository=https://test.pypi.org/legacy/
username=your-pypitest-username
password=your-pypitest-password
    • Make sure that you have the right repository configured in your ~/.pypirc file. The repository for [pypitest] should be set as repository=https://test.pypi.org/legacy/.
    • Use the --skip-existing flag if twine upload fails with HTTPError: 400 Client Error: File already exists.
    • If the above fails with HTTPError: 403 Client Error: The user '<your username>' isn't allowed to upload to project 'qcodes'., then you need to be added as Owner of qcodes project on both pypi.org and testpypi.org. This can be done by one of the core developers.
    • it can happen that twine doesn't get linked to your python environment when installed. So either fix it by manually adding it to the PATH in your environment or use full path to twine executable (it can be for example ~/.local/bin/twine)

  • check all is good in pypi (https://testpypi.python.org/pypi)

  • double check that the doi link (zenodo) in README.rst file points to the release page. Note the URL to the new zenodo "record".

  • finally release to pypi:

    twine upload -r pypi dist/*

  • Make a PR to master with the updated zenodo doi. For that update the zenodo doi link in README.md to the new one (see previous step) via a pull-request.

    • it's important that the main link is via doi.org, not zenodo.org.
    • The easiest way is to go to the webpage of the new "record" on zenodo (for example this), look at the right side of the webpage for a banner image saying DOI | <some URL>, and then click on it - a window will pop up with ways to reference the zenodo "record" page, and from there just take the links from the reStructedText section and paste them to appropriate place in the README.rst

  • Finally put an announcement with links to the changelog in Slack, Teams, and elsewhere.

Note for setting up automated release pipeline with Azure Pipelines

This is from the experience with qcodes_contrib_drivers

We had the following issues when setting up the service connection to pypi:

  • The api url suggested in the tooltip was wrong it should not be https://upload.pypi.org/legacy but https://upload.pypi.org/legacy/ (with slash at the end)
  • repository should not contain hyphens or underscores. Its just a name for the section in the pypi config file
  • While both pypi and azure service connections supports tokens, Azure service connections does not support tokens for use with pypi. It goes without saying that this is not documented :(
Add a custom footer
Add a custom sidebar
Clone this wiki locally