[QST] Version matching problem about python3.7

[sugerkiller]

I hope to install cuml in my python3.7 environment, but the RAPIDS official website only supports commands above python3.9. My environment is python3.7+cuda11.6. Can I ask which version of the package is most suitable for me?

[dantegd]

Thanks for the question @sugerkiller, the last version of RAPIDS that supported Python 3.7 was version 22.02, you can see at https://docs.rapids.ai/notices/rsn0014/. There's a chance that some later versions could be built with 3.7, but since it was dropped then, the probability of it not working from version 22.04 onwards is very high, but it might be worth a shot, I'm not sure off the top of my head what version introduced something that would break 3.7.

[sugerkiller]

I found that 21.12 is suitable for me, but there seems to be a problem when compiling to the 51st file：

-- CUDA_HOME environment variable not set.
-- CUML_BUILD_PATH environment variable not set.
-- RAFT_PATH environment variable not set.
-- Using nvcc to detect CUDA, found at C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6
-- Third party repositories have not been found so they will be cloned. To avoid this set the environment variable CUML_BUILD_PATH, containing the absolute path to the build folder where libcuml++ was built.
Found repository raft in _external_repositories/raft
-- RAFT found at: D:\Pycharm\cuml\cuml-branch-21.12\cuml-branch-21.12\python\_external_repositories\raft
E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.fetch_build_eggs(dist.setup_requires)
running install
E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
running bdist_egg
running egg_info
writing cuml.egg-info\PKG-INFO
writing dependency_links to cuml.egg-info\dependency_links.txt
writing requirements to cuml.egg-info\requires.txt
writing top-level names to cuml.egg-info\top_level.txt
Compiling cuml\cluster\agglomerative.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\cluster\dbscan.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\cluster\hdbscan.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\cluster\kmeans.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\cluster\kmeans_mg.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\common\cuda.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\string.pxd.
Compiling cuml\common\logger.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\string.pxd.
Compiling cuml\common\opg_data_utils_mg.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\common\pointer_utils.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libc\stdint.pxd.
Compiling cuml\datasets\arima.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\datasets\regression.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\decomposition\base_mg.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\decomposition\pca.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\decomposition\pca_mg.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\decomposition\tsvd.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\memory.pxd.
Compiling cuml\decomposition\tsvd_mg.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\ensemble\randomforest_common.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\ensemble\randomforest_shared.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\ensemble\randomforestclassifier.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\ensemble\randomforestregressor.pyx because it depends on E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling cuml\experimental\explainer\tree_shap.pyx because it changed.
Compiling cuml\experimental\linear_model\lars.pyx because it changed.
Compiling cuml\explainer\base.pyx because it changed.
Compiling cuml\explainer\kernel_shap.pyx because it changed.
Compiling cuml\explainer\permutation_shap.pyx because it changed.
Compiling cuml\fil\fil.pyx because it changed.
Compiling cuml\internals\internals.pyx because it changed.
Compiling cuml\linear_model\base.pyx because it changed.
Compiling cuml\linear_model\base_mg.pyx because it changed.
Compiling cuml\linear_model\elastic_net.pyx because it changed.
Compiling cuml\linear_model\lasso.pyx because it changed.
Compiling cuml\linear_model\linear_regression.pyx because it changed.
Compiling cuml\linear_model\linear_regression_mg.pyx because it changed.
Compiling cuml\linear_model\logistic_regression.pyx because it changed.
Compiling cuml\linear_model\mbsgd_classifier.pyx because it changed.
Compiling cuml\linear_model\mbsgd_regressor.pyx because it changed.
Compiling cuml\linear_model\ridge.pyx because it changed.
Compiling cuml\linear_model\ridge_mg.pyx because it changed.
Compiling cuml\manifold\t_sne.pyx because it changed.
Compiling cuml\manifold\umap.pyx because it changed.
Compiling cuml\metrics\accuracy.pyx because it changed.
Compiling cuml\metrics\cluster\adjusted_rand_index.pyx because it changed.
Compiling cuml\metrics\cluster\completeness_score.pyx because it changed.
Compiling cuml\metrics\cluster\entropy.pyx because it changed.
Compiling cuml\metrics\cluster\homogeneity_score.pyx because it changed.
Compiling cuml\metrics\cluster\mutual_info_score.pyx because it changed.
Compiling cuml\metrics\cluster\silhouette_score.pyx because it changed.
Compiling cuml\metrics\cluster\utils.pyx because it changed.
Compiling cuml\metrics\hinge_loss.pyx because it changed.
Compiling cuml\metrics\kl_divergence.pyx because it changed.
Compiling cuml\metrics\pairwise_distances.pyx because it changed.
Compiling cuml\metrics\regression.pyx because it changed.
Compiling cuml\metrics\trustworthiness.pyx because it changed.
Compiling cuml\neighbors\ann.pyx because it changed.
Compiling cuml\neighbors\kneighbors_classifier.pyx because it changed.
Compiling cuml\neighbors\kneighbors_classifier_mg.pyx because it changed.
Compiling cuml\neighbors\kneighbors_regressor.pyx because it changed.
Compiling cuml\neighbors\kneighbors_regressor_mg.pyx because it changed.
Compiling cuml\neighbors\nearest_neighbors.pyx because it changed.
Compiling cuml\neighbors\nearest_neighbors_mg.pyx because it changed.
Compiling cuml\raft\common\cuda.pyx because it changed.
Compiling cuml\raft\common\handle.pyx because it changed.
Compiling cuml\raft\dask\common\comms_utils.pyx because it changed.
Compiling cuml\raft\dask\common\nccl.pyx because it changed.
Compiling cuml\raft\include_test\raft_include_test.pyx because it changed.
Compiling cuml\random_projection\random_projection.pyx because it changed.
Compiling cuml\solvers\cd.pyx because it changed.
Compiling cuml\solvers\cd_mg.pyx because it changed.
Compiling cuml\solvers\qn.pyx because it changed.
Compiling cuml\solvers\sgd.pyx because it changed.
Compiling cuml\svm\linear.pyx because it changed.
Compiling cuml\svm\svc.pyx because it changed.
Compiling cuml\svm\svm_base.pyx because it changed.
Compiling cuml\svm\svr.pyx because it changed.
Compiling cuml\tsa\arima.pyx because it changed.
Compiling cuml\tsa\auto_arima.pyx because it changed.
Compiling cuml\tsa\holtwinters.pyx because it changed.
Compiling cuml\tsa\seasonality.pyx because it changed.
Compiling cuml\tsa\stationarity.pyx because it changed.
[ 1/79] Cythonizing cuml\experimental\explainer\tree_shap.pyx
[ 2/79] Cythonizing cuml\experimental\linear_model\lars.pyx
[ 3/79] Cythonizing cuml\explainer\base.pyx
[ 4/79] Cythonizing cuml\explainer\kernel_shap.pyx
[ 5/79] Cythonizing cuml\explainer\permutation_shap.pyx
[ 6/79] Cythonizing cuml\fil\fil.pyx
warning: cuml\fil\fil.pyx:78:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables
warning: cuml\fil\fil.pyx:79:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables
[ 7/79] Cythonizing cuml\internals\internals.pyx
[ 8/79] Cythonizing cuml\linear_model\base.pyx
[ 9/79] Cythonizing cuml\linear_model\base_mg.pyx
[10/79] Cythonizing cuml\linear_model\elastic_net.pyx
[11/79] Cythonizing cuml\linear_model\lasso.pyx
[12/79] Cythonizing cuml\linear_model\linear_regression.pyx
[13/79] Cythonizing cuml\linear_model\linear_regression_mg.pyx
[14/79] Cythonizing cuml\linear_model\logistic_regression.pyx
[15/79] Cythonizing cuml\linear_model\mbsgd_classifier.pyx
[16/79] Cythonizing cuml\linear_model\mbsgd_regressor.pyx
[17/79] Cythonizing cuml\linear_model\ridge.pyx
[18/79] Cythonizing cuml\linear_model\ridge_mg.pyx
[19/79] Cythonizing cuml\manifold\t_sne.pyx
[20/79] Cythonizing cuml\manifold\umap.pyx
[21/79] Cythonizing cuml\metrics\accuracy.pyx
[22/79] Cythonizing cuml\metrics\cluster\adjusted_rand_index.pyx
[23/79] Cythonizing cuml\metrics\cluster\completeness_score.pyx
[24/79] Cythonizing cuml\metrics\cluster\entropy.pyx
[25/79] Cythonizing cuml\metrics\cluster\homogeneity_score.pyx
[26/79] Cythonizing cuml\metrics\cluster\mutual_info_score.pyx
[27/79] Cythonizing cuml\metrics\cluster\silhouette_score.pyx
[28/79] Cythonizing cuml\metrics\cluster\utils.pyx
[29/79] Cythonizing cuml\metrics\hinge_loss.pyx
[30/79] Cythonizing cuml\metrics\kl_divergence.pyx
[31/79] Cythonizing cuml\metrics\pairwise_distances.pyx
[32/79] Cythonizing cuml\metrics\regression.pyx
[33/79] Cythonizing cuml\metrics\trustworthiness.pyx
[34/79] Cythonizing cuml\neighbors\ann.pyx
[35/79] Cythonizing cuml\neighbors\kneighbors_classifier.pyx
[36/79] Cythonizing cuml\neighbors\kneighbors_classifier_mg.pyx
[37/79] Cythonizing cuml\neighbors\kneighbors_regressor.pyx
[38/79] Cythonizing cuml\neighbors\kneighbors_regressor_mg.pyx
[39/79] Cythonizing cuml\neighbors\nearest_neighbors.pyx
[40/79] Cythonizing cuml\neighbors\nearest_neighbors_mg.pyx
[41/79] Cythonizing cuml\raft\common\cuda.pyx
[42/79] Cythonizing cuml\raft\common\handle.pyx
[43/79] Cythonizing cuml\raft\dask\common\comms_utils.pyx
[44/79] Cythonizing cuml\raft\dask\common\nccl.pyx
[45/79] Cythonizing cuml\raft\include_test\raft_include_test.pyx
[46/79] Cythonizing cuml\random_projection\random_projection.pyx
warning: cuml\random_projection\random_projection.pyx:33:4: '_Stream' redeclared
[47/79] Cythonizing cuml\solvers\cd.pyx
[48/79] Cythonizing cuml\solvers\cd_mg.pyx
[49/79] Cythonizing cuml\solvers\qn.pyx
[50/79] Cythonizing cuml\solvers\sgd.pyx
warning: cuml\solvers\sgd.pyx:271:16: Unreachable code
[51/79] Cythonizing cuml\svm\linear.pyx

Error compiling Cython file:
------------------------------------------------------------
...
from cuml.common.base import Base
from cuml.raft.common.handle cimport handle_t, _Stream
from cuml.common import input_to_cuml_array
from libc.stdint cimport uintptr_t
from libcpp cimport bool as cppbool
cimport rmm._lib.lib as rmm
       ^
------------------------------------------------------------

cuml\svm\linear.pyx:33:8: 'rmm\_lib\lib.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
                f"but got {source.shape}")
        if source.dtype != target.dtype:
            raise AttributeError(
                f"Expected an array of type {target.dtype}, "
                f"but got {source.dtype}")
        rmm.cudaMemcpyAsync(
          ^
------------------------------------------------------------

cuml\svm\linear.pyx:233:11: cimported module has no attribute 'cudaMemcpyAsync'

Error compiling Cython file:
------------------------------------------------------------
...
                f"but got {source.dtype}")
        rmm.cudaMemcpyAsync(
            <void*><uintptr_t>target.ptr,
            <void*><uintptr_t>source.ptr,
            <size_t>(source.nbytes),
            rmm.cudaMemcpyDeviceToDevice,
              ^
------------------------------------------------------------

cuml\svm\linear.pyx:237:15: cimported module has no attribute 'cudaMemcpyDeviceToDevice'

Error compiling Cython file:
------------------------------------------------------------
...
            <void*><uintptr_t>source.ptr,
            <size_t>(source.nbytes),
            rmm.cudaMemcpyDeviceToDevice,
            stream)
        if synchronize:
            rmm.cudaStreamSynchronize(stream)
              ^
------------------------------------------------------------

cuml\svm\linear.pyx:240:15: cimported module has no attribute 'cudaStreamSynchronize'
Traceback (most recent call last):
  File "setup.py", line 284, in <module>
    zip_safe=False)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\dist.py", line 989, in run_command
    super().run_command(command)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\install.py", line 84, in run
    self.do_egg_install()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\install.py", line 132, in do_egg_install
    self.run_command('bdist_egg')
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\dist.py", line 989, in run_command
    super().run_command(command)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\bdist_egg.py", line 158, in run
    self.run_command("egg_info")
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\dist.py", line 989, in run_command
    super().run_command(command)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
    cmd_obj.run()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\egg_info.py", line 318, in run
    self.find_sources()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\egg_info.py", line 326, in find_sources
    mm.run()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\egg_info.py", line 548, in run
    self.add_defaults()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\egg_info.py", line 586, in add_defaults
    sdist.add_defaults(self)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\command\sdist.py", line 113, in add_defaults
    super().add_defaults()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\command\sdist.py", line 251, in add_defaults
    self._add_defaults_ext()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\command\sdist.py", line 335, in _add_defaults_ext
    build_ext = self.get_finalized_command('build_ext')
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py", line 305, in get_finalized_command
    cmd_obj.ensure_finalized()
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\setuptools\_distutils\cmd.py", line 111, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 258, in finalize_options
    super().finalize_options()
  File "D:\Pycharm\cuml\cuml-branch-21.12\cuml-branch-21.12\python\cython_build_ext.py", line 247, in finalize_options
    **cythonize_kwargs)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Build\Dependencies.py", line 1102, in cythonize
    cythonize_one(*args)
  File "E:\Anaconda\envs\gaussian_splatting_ori\lib\site-packages\Cython\Build\Dependencies.py", line 1225, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: cuml\svm\linear.pyx

I wonder if this is because I need to install rmm first, or if it is caused by other problems.

[dantegd]

It could be a wrong version of rmm or something related, could you run the script https://github.com/rapidsai/cuml/blob/branch-24.06/print_env.sh and put the output here? That'll help us see if there's an obvious problem. Note: all the versions of RAPIDS libraries must coincide, so to build 21.12, make sure you have RMM 21.12