Install extras for testing via Tox, test with SciPy+PyPy only on Ubuntu

[bittner]

Installing extras via Conda while Tox would use Pip + virtualenv proves to be brittle. We'll factor scipy into the list of Tox environments instead, which will also make the test setup more straight-forward, less complex to understand and able to entirely run locally.

Tox environments available now (click to expand)

$ tox list
default environments:
ruff          -> Lightening-fast linting for Python
py27          -> Unit tests and test coverage
py27-scipy    -> Unit tests and test coverage
pypy27        -> Unit tests and test coverage
pypy27-scipy  -> Unit tests and test coverage
py35          -> Unit tests and test coverage
py35-scipy    -> Unit tests and test coverage
py36          -> Unit tests and test coverage
py36-scipy    -> Unit tests and test coverage
py37          -> Unit tests and test coverage
py37-scipy    -> Unit tests and test coverage
py38          -> Unit tests and test coverage
py38-scipy    -> Unit tests and test coverage
py39          -> Unit tests and test coverage
py39-scipy    -> Unit tests and test coverage
py310         -> Unit tests and test coverage
py310-scipy   -> Unit tests and test coverage
py311         -> Unit tests and test coverage
py311-scipy   -> Unit tests and test coverage
pypy38        -> Unit tests and test coverage
pypy38-scipy  -> Unit tests and test coverage
pypy39        -> Unit tests and test coverage
pypy39-scipy  -> Unit tests and test coverage
pypy310       -> Unit tests and test coverage
pypy310-scipy -> Unit tests and test coverage
package       -> Build package and check metadata (or upload package)
clean         -> Clean up bytecode and build artifacts

Note that older Python versions (2.7, 3.5, 3.6) will still be able to test with Tox, locally. Since GHA stopped supporting them in their hosted runners the burden is now on the developers, though.

SciPy requires some build prerequisites for PyPy. This is easy to fulfill on (Ubuntu) Linux, not so on the other platforms. We'll skip macOS and Windows for now.

This is a follow-up on #604.

[bittner]

@j-towns Test jobs in the GHA setup now run the tests twice, first without scipy, then with scipy installed.

This PR is ready for merging.

[j-towns]

Looks great, thanks again @bittner!

[bittner]

Do you plan to release a version 1.6.3 to PyPI now? It might even be wise to bump the version to 1.7.0 since making future an optional dependency could be seen as more than just a "patch".

It should be sufficient to push a Git tag, the rest is done by the release pipeline, e.g.

git tag 1.6.3
git push --tags

[bittner]

Do you plan to release a version 1.6.3 to PyPI now?

Any news on a release?

[j-towns]

I'll try to get around to this early next week.

[bittner]

I'll try to get around to this early next week.

Any news? -- Note that you only need to push a version tag with Git to trigger a release, now.

git checkout master && git pull
git tag 1.6.3
git push --tags

The publish Action will run, build the package cleanly and push it to PyPI (provided you've added PYPI_USERNAME and PYPI_PASSWORD to the project settings (Settings > Secrets and variables), see "Releases to PyPI" in #602 for more details).

[bittner]

@j-towns Any chance you'll get around triggering a release?

[j-towns]

Hey @bittner, I've had an exceptionally busy month - my twin sons were born on the 18th of August. I will get round to this when I can.

[bittner]

my twin sons were born on the 18th of August

Congratulations! 👶 🍼 👶 🍼

Note (again) that issuing the release should be a matter of:

1. Adding a PyPI project token as a GitHub repository secret,
2. and pushing a Git tag (git tag 1.6.3 && git push --tags).

All the rest is done automatically by the pipeline. See above for explanations, doable in really just a few minutes.

[j-towns]

Looks like I don't have access to Autograd's settings. @duvenaud could you change my role to admin?

[duvenaud]

Done.

[j-towns]

@bittner I have no knowledge of how github actions works, and I'm afraid I need clearer instructions on how to setup the publishing to PyPI. You say I need to add PYPI_USERNAME and PYPI_PASSWORD to the project settings (Settings > Secrets and variables). What values should those variables have (and should they be 'variables' as opposed to 'secrets'?)?

[bittner]

@j-towns A question aside: How do you publish the autograd package on PyPI? (Is it from your local computer?)

What values should those variables have (and should they be 'variables' as opposed to 'secrets'?)?

They should be secrets and you need to generate them on PyPI.org, like this:

1. Log in on PyPI

2. In the user menu on the top right click on "Account settings"

3. Scroll down until you read the "API tokens" section

4. Click on the Add API token button and fill in the form that opens up

5. Use the "Password" value from this token for PYPI_PASSWORD

1. The Password that is displayed needs to go into a GitHub Secret named PYPI_PASSWORD.
2. The Username (it will likely be __token__) needs to go into a GitHub Secret named PYPI_USERNAME.

For this, click on the New repository secret button in this form on GitHub:

Background Reading

This is also described in the Python Packaging docs, e.g.

To securely upload your project, you’ll need a PyPI API token. Create one at https://test.pypi.org/manage/account/#api-tokens, setting the “Scope” to “Entire account”. Don’t close the page until you have copied and saved the token — you won’t see that token again.

Hope this helps! – Let me know if you need further clarifications.

[bittner]

@j-towns Have you given configuring the two values a try?

Note that all this is only to make your life easier. You do it once, and everything is automatic after that. Instead of manual releases from your local computer you simply push a Git tag (e.g. git tag 1.6.3 and git push --tags), and GHA does the rest.

Less time wasted for maintaining your open source project, more time for your family and friends!