Does this work on Python 3.7 and later?

[dstndstn]

Describe the bug

The nbodykit documention says that The package is available for Python versions 2.7, 3.5, and 3.6., which makes users suspect that Python 3.7 is NOT supported.

Building mpsort on Python 3.6 fails in the Cython build:

# pip install mpsort
Collecting mpsort
  Using cached mpsort-0.1.17.tar.gz (88 kB)
    ERROR: Command errored out with exit status 1:
     command: /cm/shared/apps/python/python36/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lpfjdscu/mpsort_87db9512c5fe4adcbc9ff67b6a3b191b/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lpfjdscu/mpsort_87db9512c5fe4adcbc9ff67b6a3b191b/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-g0fqbq91
         cwd: /tmp/pip-install-lpfjdscu/mpsort_87db9512c5fe4adcbc9ff67b6a3b191b/
    Complete output (39 lines):
    /cm/shared/apps/python/python36/lib/python3.6/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-lpfjdscu/mpsort_87db9512c5fe4adcbc9ff67b6a3b191b/mpsort/binding.pyx
      tree = Parsing.p_module(s, pxd, full_module_name)
    performance hint: mpsort/binding.pyx:67:5: Exception check on 'myradix_u8' will always require the GIL to be acquired. Possible solutions:
    	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    	2. Use an 'int' return type on the function to allow an error code to be returned.
    performance hint: mpsort/binding.pyx:77:5: Exception check on 'myradix_i8' will always require the GIL to be acquired. Possible solutions:
    	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    	2. Use an 'int' return type on the function to allow an error code to be returned.
    performance hint: mpsort/binding.pyx:88:5: Exception check on 'myradix_u4' will always require the GIL to be acquired. Possible solutions:
    	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    	2. Use an 'int' return type on the function to allow an error code to be returned.
    performance hint: mpsort/binding.pyx:98:5: Exception check on 'myradix_i4' will always require the GIL to be acquired. Possible solutions:
    	1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    	2. Use an 'int' return type on the function to allow an error code to be returned.

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
        if 'REQUIRE_SPARSE_ALLTOALLV' in tuning:
            mpsort_mpi_set_options(MPSORT_REQUIRE_SPARSE_ALLTOALLV)

        mpsort_mpi_newarray(data.data, len(data),
                out.data, len(out),
                data.dtype.itemsize, clo.myradix,
                                        ^
    ------------------------------------------------------------

    mpsort/binding.pyx:197:36: Cannot assign type 'void (*)(const void *, void *, void *) except * nogil' to 'void (*)(void *, void *, void *) noexcept'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const void *, void *, void *) except * nogil'.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lpfjdscu/mpsort_87db9512c5fe4adcbc9ff67b6a3b191b/setup.py", line 63, in <module>
        ext_modules = cythonize(extensions)
      File "/cm/shared/apps/python/python36/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
        cythonize_one(*args)
      File "/cm/shared/apps/python/python36/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: mpsort/binding.pyx
    Compiling mpsort/binding.pyx because it changed.
    [1/1] Cythonizing mpsort/binding.pyx
    ----------------------------------------

And it seems dodgy that the mpsort repository on Github does not exist any more..... the only place it exists in on pypi??

[rainwoodman]

MPSort is at https://github.com/rainwoodman/MP-sort

The error seems to be related to the tightening of nogil annotation in Cython 3. It might be related to h5py/h5py#2268. I'll yet have to set up my dev box for investigating this further.

But a first order fix might be follow the suggestions given there? I don't think the function pointer should have checked python exceptions.

[rainwoodman]

I think this is supposed to work with relatively recent Python versions. Python 3.7 and (even likely some sort 3.8) should have been previously built on NERSC/Cori.