Gh actions for python

[BDoignies]

PR Description

This PR proposes a redesign of the CI pipeline for Python binding. Mainly, this PR changes the CI backend from Azure to Github Actions.

I am not so sure about the review of this PR though, because it was tested on my repository and that some of the changes are repository-dependent (see below for a full list).

Quick overview of changes

• Remove files of Azure pipeline (they are no longer required, but in the same time this invalidates previous CI for this PR...).
• Binding files (.h) have an extra definition to make sure it compiles on Windows. The ssize_t variable is only POSIX and not standard. Hence Windows does not have it (technically #include <BaseTsd.h> is possible, but I find it to be too heavy for a simple define).
• wrap/__init__.py -> wrap/deploy/__init__.py. Previous location caused issues with CTest. As stated in pytest good practice, import are done differently when running tests.
• New .yml files (see below)

Some 'pending changes'

• Package name in setup.py : for now it is configured as 'dgtal_githubactions_test' for testing purposes
• DGtal build options: this PR uses (for now) defaults options most of the time (except for windows where conan is necessary). A maintainer review on this would be more than helpful
• Python versions and build matrix: What python version should be supported, on what oses, ...
• Some tests commented: The goal of this PR is only about CI, not the tests themselves. Some tests did not pass not because of the setup (afaik) but because of flawed design. I commented them to make sure the workflow would work properly.

Repository-dependent changes

File pythonBindings-PR.yml is repository-independent and should run on any other fork. However the Pypi deployment is much more different.

Some documentation about the process :

• https://docs.pypi.org/trusted-publishers/
• https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
• https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment

Currently the file is setup to:

• Run on tag pushes: It extract the tagname and put it automatically as the version for the package. This might be a risk if tags are used for other reasons in the future.
• Run on an environment called 'deploy'. This is repository dependent and other names might be used in the future. Personally, I encourage maintainers to use an environment that requires 'manual' validation for build and upload to pypi.
• Push the wheel to TestPypi which is on the test pypi and owned by me (sorry for 'stealing the name', is here for test purposes only and will be removed when this PR is merged of if asked by owners of the library). Before changing the links, make sure Trusted publishers is setup for this repository.

What I did not test :

• MacOS and Windows pip install : https://test.pypi.org/manage/project/dgtal-ghactions-test/releases/. I'd appreciate if anyone could test that as I did not specifically asked for static linking of libraries (linux manywheel exports libraries inside the wheel).

What's missing:

• Documentation: I did not rewrite the documentation yet. I am waiting for any comments reviewers might have.

• Github releases: It is possible to publish directly the .wheel and make them available as part of the tags.

• Comments and reviews: I'm am neither an expert of Dgtal nor an expert of CI and python build. Any comments is welcome to make sure everything is fine and for the best 😺 !

Checklist

Most of the following are not applicable to this PR.

• [?] Unit-test of your feature with Catch.
• [?] Doxygen documentation of the code completed (classes, methods, types, members...)
• [?] Documentation module page added or updated.
• New entry in the ChangeLog.md added.
• [?] No warning raised in Debug mode.
• [?] All continuous integration tests pass (Github Actions)

[dcoeurjo]

Many thanks for the PR. Just a first test, I got a Python.h error on my Mac:

/Users/davidcoeurjolly/Sources/DGtal/build/_deps/pybind11-src/include/pybind11/detail/common.h:213:10: fatal error: 'Python.h' file not found
#include <Python.h>
         ^~~~~~~~~~

It reminds me something similar for UTK, I'd need to investigate a bit.

[dcoeurjo]

Ok something I should debug:

-- Found PythonInterp: /Users/davidcoeurjolly/anaconda3/python.app/Contents/MacOS/python (found version "3.11.4")
-- Found PythonLibs: /Users/davidcoeurjolly/anaconda3/lib/libpython3.11.dylib
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Performing Test HAS_FLTO_THIN
-- Performing Test HAS_FLTO_THIN - Success
-- Tests for Python-only ENABLED
-- Configuring done (4.2s)
CMake Error in wrap/CMakeLists.txt:
  Imported target "pybind11::module" includes non-existent path

    "/Users/davidcoeurjolly/anaconda3/python.app/Contents/include/python3.11"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.



CMake Error in wrap/CMakeLists.txt:
  Imported target "pybind11::module" includes non-existent path

    "/Users/davidcoeurjolly/anaconda3/python.app/Contents/include/python3.11"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

[dcoeurjo]

(ok problem solved.. just a conda issue)

[dcoeurjo]

Why such string statement ?

[BDoignies]

This the pythonic way of doing multiline comments. As mentionned in the PR, those test do not run -- and this is not because of GH actions.

[dcoeurjo]

Very nice PR, thx @BDoignies . We should discuss a bit about this pypi thing..

[dcoeurjo]

In the build directory, how can I just test the python binding ? Just import dgtal in this folder?

[BDoignies]

You can follow the ".github/workflows/pythonBindings-PR.yml" from line 92 and onwards if you have any more questions about the test process. Here are the lines answering your question :

In the build directory, run

1. ctest -R python -C ${{ matrix.BUILD_TYPE }} -V --output-on-failure

You can also test that the module can be properly imported with the line :

2. python -c "import dgtal"

[BDoignies]

We should discuss a bit about this pypi thing..

You know where to find / contact me to setup a meeting for this !

[dcoeurjo]

You know where to find / contact me to setup a meeting for this !

I will ;)

[dcoeurjo]

Merging to test the deploy and GH

[BDoignies]

Just to make sure information is somewhere in case it is needed.

Reasons for Pypi deployment to break:

• Tags are not appropriate version numbers. Fix: either change tag or update the workflow file to filter and extract tag information properly
• Same version / already pushed files. Fix: none. You simply can not push twice the same version.
• manylinux version. Because DGtal may use advanced features, the manylinux wheel should use relatively new versions. If only linux deployment breaks, check the autiwhell call for the version.