Skip to content

How to Package a New RMG Release

Jump to bottom
Sevy Harris edited this page Aug 8, 2023 · 26 revisions

Release Numbering

RMG-Py and RMG-database releases can be different versions, and a new database release does not have to coincide with an RMG-Py release. However, because RMG-Py and RMG-database development is often closely linked, it is a good idea to make releases for both on the same schedule.

Release numbering should be updated based on how much the code has shifted. We currently use a naming system like so

RMG-Py vX.Y.Z

very loosely following the concept of semantic versioning, e.g. Z updates should contain bugfixes, Y updates should contain feature level changes, and X updates should contain backward-incompatible changes.

Git Workflow

We have incorporated some elements of Git Flow into our workflow. There are now separate stable and main branches. Releases are tagged on the stable branch, which is only updated for releases. The main branch is essentially the develop branch and contains all ongoing work. Releases should start as a new branch created off of main, upon which release related changes are made, such as updating the version number and documentation. The release branch should then be merged into the stable branch, creating the release. The stable branch should then be merged back into main to ensure that changes associated with the release are incorporated, and that the version tag is also visible on main.

The primary benefit of this workflow is that hotfixes to released versions are possible. For example, if development has already progressed on the main branch when a major bug is discovered, we can fix the bug and push a patch release without being forced to include the latest developments into the release. To do so, the hotfix branch should be created from stable, the fixes applied, and then merged back into stable to create the release. The stable branch can then be merged into main again to ensure that the changes are on main.

Release Creation

  1. When RMG-Py and/or RMG-database are ready for a new release, first create release branches off of main, i.e. vX.Y.Z. Development can continue on the main branches after the release branch has been created.
  2. Update all version numbers first, because a lot of the other files and functions queries these version numbers.
    • For RMG-Py: RMG-Py/rmgpy/version.py, RMG-database version requirement in RMG-Py/.conda/meta.yaml if necessary
    • For RMG-database: RMG-database/setup.py
    • For PyDQED: PyDQED/setup.py, PyDQED/pydqed.pyx
    • For PyDAS: PyDAS/pydas/version.py
  3. Create new release notes in the documentation for the version: RMG-Py/documentation/source/users/rmg/releaseNotes.rst
  4. Make sure main copyright stamps to reflect year change if necessary in RMG-Py/LICENSE.txt. If an updated is needed, make the change in the LICENSE.txt file and then run RMG-Py/utilities.py update-headers. Make sure to review the changes in case there are any files which were processed incorrectly. Here's a script to help look for any files that changed more than just the one line:
#!/bin/bash
# Create a reference.txt file that contains an example git diff
# with just the one line changed. This can be done with:
git diff Arkane.py > example_file.txt
tail -n 8 example_file.txt > reference.txt

# Also make a file with the list of changed files:
git diff --name-only > files_changed.txt

my_file="files_changed.txt"
lines=$(cat $my_file)

echo "These files have more than one line changed (more than just the date in the license was changed)"
for line in $lines
do
    git diff $line > file_diff.txt
    tail -n 8 file_diff.txt > file_diff_trimmed.txt
    result=$(diff file_diff_trimmed.txt reference.txt)
    rm file_diff.txt
    rm file_diff_trimmed.txt
    if [ ! -z "$result" ]
        then echo $line
    fi
    if [[ $(git diff $line | wc -l) -ne 13 ]];
        then echo $line
    fi
done

# cleanup extra files
rm example_file.txt
rm reference.txt
rm files_changed.txt

  1. Compile new PDF documentation from source for the release by going into documentation folder and using the command make latexpdf. If you have difficulty doing that you can alternatively run make latex to generate the latex files and run the latex using Overleaf or another software. Replace the previous documentation pdfs in the RMG-Py/documentation folder. Create a commit that states that PDF documentation for release X.Y.Z has been updated. (Note that RMG-Py, PyDAS, and PyDQED repos all have documentation. RMG-database does not.)

  2. Check that everything works correctly. Unit tests are automatically run by Travis. RMG-tests is also automatically triggered. Confirm that both pass without issues by pushing the release branch to GitHub. You may want to test that the Anaconda binary for the repo compiles correctly at this point in time and that the packages run properly on a clean anaconda environment.

    • Example testing procedure in Mac and Linux: 
      conda create --name rmg_test -c rmg rmg=X.Y.Z
source activate rmg_test
rmg.py input.py
source deactivate

  3. Once you're sure that the release is ready, then it can be merged into the stable branch. This can either be done locally or via a GitHub pull request. If doing locally, switch to the stable branch and run git merge --no-ff vX.Y.Z -m "RMG-Py vX.Y.Z release". Modify the commit message appropriately for other repos. If doing via GitHub, be sure to modify the merge commit message.

  4. Tag the new release on your local stable and then push the tagged branch to GitHub (this is necessary because the GitHub Actions workflow reads the tag to assign the version number). Go to the releases tab of the repository and click the 'Draft a new release' button. Create the release title RMG-Py vX.Y.Z.

  5. Once the changes have been pushed to the stable branches on GitHub, GitHub Actions will automatically compile and upload the binaries for Linux-64 and OSX-64. If there are issues with the OSX binaries they can be built locally and copied over to the RMG channel. The OSX binaries are sensitive to the CONDA_BUILD_SYSROOT specification in the conda_build_config.yaml, the appropriate path can be found by running xcrun --show-sdk-path.

    • Important Note: If updating the version requirement of RMG-database in RMG-Py, the new binary for RMG-database must be uploaded before the RMG-Py binary is created. Generally, this just means that you should push to RMG-database stable before RMG-Py stable in order to give Travis time to finish.
    • Automatic uploading of the binaries to Anaconda.org is enabled by an authentication token called ANACONDA_TOKEN. The current token will expire on 4/26/24, at which point an new token will need to be generated (either via Anaconda.org or from the command line).

  6. Some binary version of RMG dependencies which have an expiration date need also be updated. So far we have mopac to be updated.

  7. If the release had substantial updates, email this news to the relevant RMG user base along with the release notes and links for how to install. Reference older release emails for details. (Key groups: rmg-users mailing list, rmg_dev@mit.edu, Green Group, Comocheng Group, any users who have been emailing us in recent history).

  8. Don't forget to merge stable back into main at the end so that future releases built off main can be merged into stable without issue!

Items which are now automatically updated (but should be double checked):

Home

RMG

Anaconda

Github/Travis

Clone this wiki locally