BUG: import fails on Python 3.10 (mac-os only!)

[staticdev]

Describe the issue:

When I try to import numpy on mac-os using Py3.10 I get:

E   RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, more information is available at https://numpy.org/doc/stable/user/building.html#accelerated-blas-lapack-libraries Otherwise report this to the vendor that provided NumPy.
E   RankWarning: Polyfit may be poorly conditioned

Reproduce the code example:

import numpy as np

Error message:

Complete runs can be checked here: https://github.com/staticdev/irpf-cei/runs/4266824409?check_suite_focus=true


tests/test_cei.py:5: in <module>
    import numpy as np
.nox/tests-3-10/lib/python3.10/site-packages/numpy/__init__.py:391: in <module>
    raise RuntimeError(msg)
E   RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, more information is available at https://numpy.org/doc/stable/user/building.html#accelerated-blas-lapack-libraries Otherwise report this to the vendor that provided NumPy.
E   RankWarning: Polyfit may be poorly conditioned

### NumPy/Python version information:

1.21.4 3.10.0 (default, Nov 22 2021, 15:51:54) 

[charris]

Where did you get NumPy?

[mattip]

It seems your CI run https://github.com/staticdev/irpf-cei/runs/4266824409 somehow is getting numpy, but it does not state where or how. Can you make the pip install step more transparent?

[staticdev]

I get numpy on Github Actions from PyPI. Though, I wasn't able to reproduce the error on Macbook using Python 3.10.0 and same version of numpy. Even when I don't have numpy imports, pandas imports indirectly generate the same errors https://github.com/staticdev/irpf-cei/runs/4291736117?check_suite_focus=true

This is a very strange error that happens specifically on macos + python 3.10 I tried other combinations (using python3.9 and linux/windows). Any idea of how can I provide more info?

[staticdev]

BTW @charris @mattip I found two references saying that 1.19 version caused this error on macos.
https://stackoverflow.com/questions/64956168/macos-big-sur-python3-cannot-import-numpy-due-to-polyfit-error
pypa/cibuildwheel#402

It also mentions a closed issue that could be considered a dupe from this one, but it was closed without fix: #15947

[seberg]

It has been a while since we had issues like this reported last, I think. One thing that sometimes happened was that Accelerate was loaded (e.g. by ipython or Qt?) before NumPy was imported. And then NumPy would pick that up as its BLAS backend.

But, I feel our wheels should be immune to that, so probably your problem is that your CI ends up compiling NumPy from scratch (through pip) and that finds the system cblas which is provided by the buggy Accelerate.

[staticdev]

@seberg any idea how can I workaround this? For now I set macos Python to 3.9 and CI/CD works for testing. But I would really like to have it running on 3.10.

[charris]

@staticdev It isn't a bug, all that changed was a check for a known bad library, Accelerate, and the cure is to not use Accelerate. Numpy 1.21 now has wheels for Python 3.10 and should be used instead. Perhaps one of the other packages is requiring an earlier version? You might try upgrading Pandas.

[charris]

Also be sure that you install NumPy with pip to use python -mpip install numpy, where python is the version you intend to use. A bare pip is unreliable for that.

[staticdev]

@staticdev It isn't a bug, all that changed was a check for a known bad library, Accelerate, and the cure is to not use Accelerate. Numpy 1.21 now has wheels for Python 3.10 and should be used instead. Perhaps one of the other packages is requiring an earlier version? You might try upgrading Pandas.

For sure this is not the case, actually I use pip install requirements from exported requirements from poetry.lock, which is ensuring latest numpy:
https://github.com/staticdev/irpf-cei/blob/8e7929f7b508502a34f1e709719035a7b3479441/poetry.lock#L1521

And also, I am using the latest pandas 1.3.4 BTW.

[Deuchnord]

Having the same issue with GitHub workflow on Python 3.10 and macos-latest.
It seems to be related to Poetry, because I don't have the problem with Pipenv:

• Pipenv, macos-10.15 (== macos-latest): https://github.com/Kosmorro/lib/runs/4408984359
• Poetry, macos-latest: https://github.com/Kosmorro/lib/runs/4408710888

[staticdev]

@Deuchnord interesting, so it seems to be a very specific combination of macos-latest from GA (since I tested in a Mac and it does not happen); poetry; numpy > 1.19 AND python 3.10.0 (doesn't happen on 3.9). These are the ingredients for failure I might say. Still a puzzle why.

[Deuchnord]

Didn't see actions/runner-images#4060 until now, but looks like macos-latest will move to macOS 11 tomorrow. We may want to wait until that to if it fixes our issue :)

[staticdev]

@Deuchnord retested with macos-latest from GA again today. It is working! I will close then.