Wheels compatible with Numpy 2.0

[skoudoro]

Hi cvxpy Team,

Do you have any wheels compatible with the future numpy 2.0 ?

Currently, when I compile my package with Numpy 2.0.0dev and try to run it with cvxpy, I encounter the following error:

import _cvxcore
RuntimeError: module compiled against ABI version 0x1000009 but this version of numpy is 0x2000000

Numpy 2.0 is coming and I would like to make sure everything will be smooth with the transition.

Any advice will be welcomed.

Thanks !

[Transurgeon]

Hello @skoudoro thanks for opening this issue. I believe cvxpy has already added the ruff rule NPY201 mentioned in the NumPy docs. I will try testing against the main branch today. Is this the right place to get the updated version of NumPy 2.0?

[skoudoro]

Is this the right place to get the updated version of NumPy 2.0?

Yes, we also use it to put our wheels for Numpy 2.0. You will need to do a request to scientific-python community if you would like to upload the one for cvxpy.

Thank you for your feedback.

[Transurgeon]

Some updates on this. I tried testing with Numpy 2.0 but I also get the following error:

  import _cvxcore
RuntimeError: module compiled against ABI version 0x1000009 but this version of numpy is 0x2000000

I first got the nightly build from scipy which created some conflicts with OSQP (since it requires scipy <=1.12).
I can try contacting some folks at scientific python to ask for advice.

@phschiele @PTNobel any thoughts on this issue? I am not too familiar with the process of creating new wheels for cvxpy.

[PTNobel]

@Transurgeon did you try deleting artifacts before installing cvxpy locally?

[Transurgeon]

@Transurgeon did you try deleting artifacts before installing cvxpy locally?

No, I am not sure what artefacts are and where to delete them. Also when I'm installing cvxpy locally, the nightly NumPy version I installed gets overridden by the oldest-supported-numpy. Do we need to update the pyproject.toml setup file to be able to create new wheels?

[rileyjmurray]

@Transurgeon @PTNobel this issue might be related to the numpy swig interface file we use:

https://github.com/cvxpy/cvxpy/blob/master/cvxpy/cvxcore/python/numpy.i.

Perhaps we need an updated version of this file?

[Transurgeon]

@skoudoro Some more updates. I was able to generate wheels for cvxpy with the nightly version of NumPy and SciPy (see my PR linked above).
However, it seems like the CI builds are failing due to those packages not being available through pip. Is there a way to test CI builds with the nightly versions of NumPy? (I think we would need a different version of the pyproject.toml setup file which may cause some problems).
I believe you can also test if the build works fine for your package by checking out to the branch of the PR above and installing cvxpy from source.

[skoudoro]

Great ! Thank you for the update @Transurgeon!

However, it seems like the CI builds are failing due to those packages not being available through pip

we use the extra-index-url option of pip and provide the scientific-python url. This allow to search in pypi and other repo

I believe you can also test if the build works fine for your package by checking out to the branch of the PR above and installing cvxpy from source.

ok, I might try later this week

[h-vetinari]

FWIW, I'm encountering compilation errors with cxpy 1.5.1 against numpy 2.0.0rc2:

  $PREFIX/lib/python3.10/site-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
  cvxpy/cvxcore/python/cvxcore_wrap.cxx: In function 'PyArrayObject* obj_to_array_no_conversion(PyObject*, int)':
  cvxpy/cvxcore/python/cvxcore_wrap.cxx:3788:62: error: cannot convert 'PyObject*' {aka '_object*'} to 'const PyArrayObject*' {aka 'const tagPyArrayObject_fields*'}
   3788 |                             PyArray_EquivTypenums(array_type(input), typecode)))
        |                                                              ^~~~~
        |                                                              |
        |                                                              PyObject* {aka _object*}
  cvxpy/cvxcore/python/cvxcore_wrap.cxx:3725:51: note: in definition of macro 'array_type'
   3725 | #define array_type(a)          (int)(PyArray_TYPE(a))
        |                                                   ^
  $PREFIX/lib/python3.10/site-packages/numpy/_core/include/numpy/ndarraytypes.h:1563:35: note:   initializing argument 1 of 'int PyArray_TYPE(const PyArrayObject*)'
   1563 | PyArray_TYPE(const PyArrayObject *arr)
        |              ~~~~~~~~~~~~~~~~~~~~~^~~

[PTNobel]

@h-vetinari

I believe the current plan is:

1. 1.5.0 and 1.5.1 do not support NumPy 2.0
2. When NumPy 2.0 becomes stable and we can easily build against it in CI, we will merge Updating numpy.i (cvxcore) swig interface #2392 and release 1.5.2.

[h-vetinari]

When NumPy 2.0 becomes stable

The release candidates are as stable as you can hope for, the only reason they're not GA is to allow projects to prepare, rather than break the world. So I would recommend quite highly to publish 1.5.2 already based on numpy 2.0.0rc2.