Tests fail: RuntimeError: Compiled extensions are unavailable.

[yurivict]

Problem description

Tests fail like this:

gensim/corpora/indexedcorpus.py:14: in <module>
    from gensim import interfaces, utils
gensim/interfaces.py:19: in <module>
    from gensim import utils, matutils
gensim/matutils.py:1354: in <module>
    raise utils.NO_CYTHON
E   RuntimeError: Compiled extensions are unavailable. If you've installed from a package, ask the package maintainer to include compiled extensions. If you're building Gensim from source yourself, install Cython and a C compiler, and then run `python setup.py build_ext --inplace` to retry.

The project was built using PEP517 compliant build and install modules, and cython is available.
It isn't clear why compiled extensions aren't available.

Steps/code/corpus to reproduce

n/a

Versions

Please provide the output of:

import platform; print(platform.platform())
import sys; print("Python", sys.version)
import struct; print("Bits", 8 * struct.calcsize("P"))
import numpy; print("NumPy", numpy.__version__)
import scipy; print("SciPy", scipy.__version__)
import gensim; print("gensim", gensim.__version__)
from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)

FreeBSD-13.2-STABLE-amd64-64bit-ELF
Python 3.9.17 (main, Aug  6 2023, 01:11:50) 
[Clang 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc1238
Bits 64
NumPy 1.25.0
SciPy 1.11.1
gensim 4.3.2
FAST_VERSION 1

[mpenkov]

The project was built using PEP517 compliant build and install modules, and cython is available.

Can you please expand on this? It looks like you're building from source, so you started by cloning (or downloading) the gensim source code. What steps did you take after that?

[yurivict]

We download the code from PYPI gensim-4.3.2.tar.gz
Then we run build:

cd /usr/ports/textproc/py-gensim/work-py39/gensim-4.3.2 && /usr/bin/env XDG_DATA_HOME=/usr/ports/textproc/py-gensim/work-py39  XDG_CONFIG_HOME=/usr/ports/textproc/py-gensim/work-py39  XDG_CACHE_HOME=/usr/ports/textproc/py-gensim/work-py39/.cache  HOME=/usr/ports/textproc/py-gensim/work-py39 PATH=/usr/local/libexec/ccache:/usr/ports/textproc/py-gensim/work-py39/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/textproc/py-gensim/work-py39/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  "  MANPREFIX="/usr/local" CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" /usr/local/bin/python3.9 -m build --no-isolation --wheel 

Then we run the install command:

cd /usr/ports/textproc/py-gensim/work-py39/gensim-4.3.2 && /usr/bin/env XDG_DATA_HOME=/usr/ports/textproc/py-gensim/work-py39  XDG_CONFIG_HOME=/usr/ports/textproc/py-gensim/work-py39  XDG_CACHE_HOME=/usr/ports/textproc/py-gensim/work-py39/.cache  HOME=/usr/ports/textproc/py-gensim/work-py39 PATH=/usr/local/libexec/ccache:/usr/ports/textproc/py-gensim/work-py39/.bin:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin PKG_CONFIG_LIBDIR=/usr/ports/textproc/py-gensim/work-py39/.pkgconfig:/usr/local/libdata/pkgconfig:/usr/local/share/pkgconfig:/usr/libdata/pkgconfig MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector-strong -fno-strict-aliasing "  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector-strong " LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector-strong -fno-strict-aliasing  "  MANPREFIX="/usr/local" CCACHE_DIR="/tmp/.ccache" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 0644"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" /usr/local/bin/python3.9 -m installer --destdir /usr/ports/textproc/py-gensim/work-py39/stage --prefix /usr/local /usr/ports/textproc/py-gensim/work-py39/gensim-4.3.2/dist/gensim-4.3.2*.whl

[mpenkov]

Thank you. Please provide full command line output of your build and install commands.

Also, please let us know if it is possible to reproduce this with a previous version of gensim (4.3.1).

[mpenkov]

@yurivict Are you able to provide the requested information? If not, then we cannot help you.