Add scheduled workflow to run tests with unpinned dependencies

[matt-graham]

Fixes #1161

Adds a new workflow plus associated tox environment to run tests with unpinned dependencies on latest stable Python version once a week (Satuday at midnight). This currently is set up to run on GitHub hosted runners to allow testing across multiple operating systems and uses pytest-xdist plug-in to parallelise across available cores on the runner, rather than creating a matrix to parallelise across test files as in the main tests workflow. It is not clear if the full tests will complete in the 6 hours maximum GitHub Actions job time on GitHub hosted runners (may do though given Ubuntu runners now have 4 cores by default and I think the MacOS runners have 3 cores).

Questions:

• Do we want to test against a specific Python version (3.11?) rather than or in addition to latest stable Python version?
• Is once a week the appropriate frequency? Possibly once a month would suffice?
• Should this be done on self-hosted runners instead?

[tamuri]

Do we want to test against a specific Python version (3.11?) rather than or in addition to latest stable Python version?

This workflow should be our canary in the coal mine - latest stable Python & packages. To that end, we state pandas~=2.0.0 in the pyproject. Can we ignore that too?

Is once a week the appropriate frequency? Possibly once a month would suffice?

Once a month is probably okay. Don't want to get too distracted.

Should this be done on self-hosted runners instead?

We'd only be targetting Linux in that case. I do like the the extensive coverage we get if we use GH runners.

[matt-graham]

This workflow should be our canary in the coal mine - latest stable Python & packages. To that end, we state pandas~=2.0.0 in the pyproject. Can we ignore that too?

Agree it would be worth testing against unpinned pandas versions as this is probably most likely dependency we'll get breaking changes in. We could remove the version specified for pandas from dependencies in pyproject.toml altogether or relax ~=2.0.0 to >=2, ~=2.0 or similar. While this would then mean the new workflow here tests against latest stable pandas version, it would also mean that it becomes more of a pain to update the requirements files with pip-compile, as without the version specifier pip-compile would also I think resolve to the latest stable pandas version at the time run, and currently the tlo package will not work with pandas v2.1 or newer I believe. It looks like this would be relatively easy to workaround by using the --upgrade--packages argument to pip-compile to specify a constraint on the pandas versions though. Altermatively we could install the dependencies manually in the new tox environment, and force it to not ignore / not install package dependencies by updating the install_command option with --no-deps but this feels a bit hacky and requires us to keep the base requirements in two different places.

Once a month is probably okay. Don't want to get too distracted.

Will update.

We'd only be targetting Linux in that case. I do like the the extensive coverage we get if we use GH runners.

👍