Skip to content

Commit

Permalink
[INFRA] Proposal for building stable docs in remote runner for releas…
Browse files Browse the repository at this point in the history
…es (#4160)

* Add workflow for stable docs

* Add to readme

* Add configs for testing

* Use specific version tag

* Update from comments

* Remove test recipe

* Update readme

* Update comments

* Add ssh configs

* Update maintenance docs

* Update maintenance docs

* Syntax

* Add note

* Formatting
  • Loading branch information
ymzayek committed Jan 2, 2024
1 parent 2504bed commit 6dc25ec
Show file tree
Hide file tree
Showing 4 changed files with 113 additions and 8 deletions.
10 changes: 10 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ $ git commit -m "[example] plot_nilearn_101.py"

However for quick checks to do yourself you should always opt for local builds following the instructions here: [building-documentation](https://nilearn.github.io/stable/development.html#building-documentation).

Note: setuptools needs to be installed to run the doc build with python 3.12.

#### Dataset caching

We also implemented a dataset caching strategy within this Actions workflow such that datasets are only downloaded once every week. Once these datasets are cached, they will be used by all jobs running on Actions without requiring any download. This saves a lot of time and avoids potential network errors that can happen when downloading datasets from remote servers.
Expand Down Expand Up @@ -107,6 +109,14 @@ This workflow will fail if it finds any potential target to be converted.

Sorts Python imports alphabetically and by section. Configured in [pyproject.toml](../../pyproject.toml)

## Building the stable release documentation

### release-docs.yml

Should be triggered automatically after merging and tagging a release PR to
build the stable docs with a GitHub runner and push to nilearn.github.io.
Can Also be triggered manually.

## Running unit tests

### test_with_tox.yml
Expand Down
81 changes: 81 additions & 0 deletions .github/workflows/release-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
name: Build release docs

on:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# This will run automatically after tagging the release commit on main
push:
branches:
- main
tags:
- '*'

jobs:
release-docs:
if: github.repository == 'nilearn/nilearn'
runs-on: ubuntu-latest
timeout-minutes: 360
env:
BROWSER: /usr/bin/firefox
DISPLAY: :99.0
NILEARN_DATA: /home/runner/work/nilearn/nilearn/nilearn_data
defaults:
run:
shell: bash -el {0}

steps:
- name: Checkout nilearn
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Add SSH key
env:
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
run: |
mkdir -p ~/.ssh
ssh-keyscan github.com >> ~/.ssh/known_hosts
echo "${{ secrets.ACTIONS_SSH_DEPLOY }}" > ~/.ssh/github_actions
chmod 600 ~/.ssh/github_actions
ssh-agent -a $SSH_AUTH_SOCK > /dev/null
ssh-add ~/.ssh/github_actions
- name: Install apt packages
run: |
sudo -E apt-get -yq update
sudo -E apt-get -yq --no-install-suggests --no-install-recommends install \
dvipng texlive-latex-base texlive-latex-extra
- name: Setup python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install packages
run: |
python -m pip install --user --upgrade pip setuptools
python -m pip install .[plotting,doc]
# Update the authors file and the names file
# in case a contributor has been added to citation.cff
# but did not run the maint_tools/citation_cff_maint.py script.
- name: update AUTHORS.rst and doc/changes/names.rst
run: python maint_tools/citation_cff_maint.py

# Set up and launch a virtual browser needed for one example to run
# without stalling the job. The example launches an html in the browser.
- name: Set up display server for virtual browser
run: Xvfb -ac :99 -screen 0 1280x1024x16 > /dev/null 2>&1 &

- name: Build docs
env:
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
run: |
git config --global user.email "actions@github.com"
git config --global user.name "GitHub actions"
cd doc
set -o pipefail
export VERSIONTAG=$(git describe --tags --abbrev=0)
make install
6 changes: 3 additions & 3 deletions doc/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ SPHINXOPTS = -v
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
VERSIONTAG = x.y.z

# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
Expand Down Expand Up @@ -173,6 +172,7 @@ check:
BUILD_DEV_HTML=1 $(SPHINXBUILD) -W --keep-going -T -n -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html

install:
@echo $(VERSIONTAG)
rm -rf _build/doctrees _build/nilearn.github.io
# first clone the nilearn.github.io repo because it may ask
# for password and we don't want to delay this long build in
Expand All @@ -190,5 +190,5 @@ install:
git add -A && \
git add .nojekyll && \
git commit -a -m 'Make install' && \
git tag $(VERSIONTAG)
git push --tags
git tag $(VERSIONTAG) && \
git push $(VERSIONTAG)
24 changes: 19 additions & 5 deletions doc/maintenance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,14 @@ Once the PR has been reviewed and merged, pull from master and tag the merge com
When building the distribution as described below, ``hatch-vcs``, defined in ``pyproject.toml``,
extracts the version number using this tag and writes it to a ``_version.py`` file.

Build of stable docs
--------------------

Once the new version tag of a release is pushed upstream following the step
above, the Github Actions workflow ``release-docs.yml`` will be triggered
automatically to build the stable docs and push them to
our github pages repository ``nilearn/nilearn.github.io``. The workflow can
also be triggered from the Actions tab.

Build the distributions and upload them to Pypi
-----------------------------------------------
Expand Down Expand Up @@ -337,8 +345,14 @@ and edit the ``x.y.z`` tag by providing a description,
and upload the distributions we just created (you can just drag and drop the files).


Build and deploy the documentation
----------------------------------
Build and deploy the documentation manually
-------------------------------------------

.. note::

This step is now automated as described above. If there is a need to run it
manually please follow the instructions below.


Before building the documentation, make sure that the following LaTeX
dependencies are installed on your system:
Expand Down Expand Up @@ -372,13 +386,13 @@ See available linux distributions of texlive-latex-base and texlive-latex-extra:
- https://pkgs.org/search/?q=texlive-latex-base
- https://pkgs.org/search/?q=texlive-latex-extra

We now need to update the documentation. Make sure to change ``x.y.z`` to the
current release version:
We now need to update the documentation.

.. code-block:: bash
cd doc
make install VERSIONTAG=x.y.z
export VERSIONTAG=$(git describe --tags --abbrev=0)
make install
This will build the documentation (beware, this is time consuming...)
Expand Down

0 comments on commit 6dc25ec

Please sign in to comment.