[Installation]: Getting requirements to build wheel did not run successfully.

[ashishpatel26]

Installation check

• I have read the installation guide.

Installation OS

Windows

Installation Method

CloudFormation

Kubernetes Cluster Type

N/A - Docker

MLRun CE Version

Latest

Issue Description

Getting requirements to build wheel did not run successfully.

Installation Logs

error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [62 lines of output]
C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\config\setupcfg.py:293: _DeprecatedConfig: Deprecated config in setup.cfg
!!

          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.

          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.

          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************

  !!
    parsed = self.parsers.get(option_name, lambda x: x)(value)
  running egg_info
  writing lib3\PyYAML.egg-info\PKG-INFO
  writing dependency_links to lib3\PyYAML.egg-info\dependency_links.txt
  writing top-level names to lib3\PyYAML.egg-info\top_level.txt
  Traceback (most recent call last):
    File "C:\Users\ibm26\.conda\envs\mlops\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 351, in <module>
      main()
    File "C:\Users\ibm26\.conda\envs\mlops\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 333, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\ibm26\.conda\envs\mlops\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel        
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
      self.run_setup()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
      exec(code, locals())
    File "<string>", line 271, in <module>
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
      return run_commands(dist)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
      dist.run_commands()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\dist.py", line 1233, in run_command
      super().run_command(command)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
      cmd_obj.run()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 319, in run
      self.find_sources()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 327, in find_sources
      mm.run()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 549, in run
      self.add_defaults()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 587, in add_defaults
      sdist.add_defaults(self)
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\command\sdist.py", line 113, in add_defaults
      super().add_defaults()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\command\sdist.py", line 251, in add_defaults
      self._add_defaults_ext()
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\command\sdist.py", line 336, in _add_defaults_ext     
      self.filelist.extend(build_ext.get_source_files())
    File "<string>", line 201, in get_source_files
    File "C:\Users\ibm26\AppData\Local\Temp\pip-build-env-s7lia8l4\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Additional Information

No response

[liranbg]

I am not sure i understand what you have tried to do. please do elaborate
mention your python version, pip version, command invoked

[fschlz]

@liranbg the issue is that mlrun depends on pyyaml==5.4.1, but that version of the package doesn't comply with PEP-517.
And trying to install with a higher pyyaml version fails.

I got a similar issue, running Python 3.10.12, and using poetry to manage dependencies:

  • Installing pyyaml (5.4.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel
  
  running egg_info
  writing lib3/PyYAML.egg-info/PKG-INFO
  writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
  writing top-level names to lib3/PyYAML.egg-info/top_level.txt
  Traceback (most recent call last):
    File "/usr/local/Cellar/poetry/1.7.1/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/usr/local/Cellar/poetry/1.7.1/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/Cellar/poetry/1.7.1/libexec/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 271, in <module>
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 321, in run
      self.find_sources()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
      mm.run()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 551, in run
      self.add_defaults()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
      sdist.add_defaults(self)
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/command/sdist.py", line 112, in add_defaults
      super().add_defaults()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
      self._add_defaults_ext()
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
      self.filelist.extend(build_ext.get_source_files())
    File "<string>", line 201, in get_source_files
    File "/var/folders/fq/5j_lg3lj0bqbk063l9qgg0b80000gn/T/tmp19ve21ez/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources

I saw that Python versions 3.8 and 3.9 are specifically supported, but trying to install mlrun with Python 3.10 still worked, until one gets to the pyyaml.
After retrying with Python 3.9 the issue subsided. However, installing with a Python version that is not supported should just throw an error.

Perhaps a solution here would be

1. to remove the pyyaml dependency in mlrun and use the python built-in yaml package.
   I've seen pyyaml cause similar dependency issues in other projects. Hasn't been worth keeping it around, in my experience.
2. Currenlty, mlrun also still defines dependencies in a requirements.txt file. Switching to a complete project definition in pyproject.toml might alleviate some issues overall. Then one might also remove the necessity of the setup files. Then define a broader range of allowed Python versions (i.e. python = ">=3.8,<=3.12")

[liranbg]

Hi @fschlz - great findings and thanks for the provided analysis. while tackling (1) is easier, I will provide a PR resolving it
about (2) - that requires a bit more of an effort and will be resolved in the coming versions as part of how we refactor mlrun client/server packages.

P.S
@fschlz - why do you assume there is a built-in yaml package for Python?
also see python-poetry/poetry#8287 (I see you also used poetry, so perhaps related)

[fschlz]

@liranbg I cam back to this today and the issue still prevails.
Any plans on upgrading to the latest pyyaml version?

PS: It seems I was def wrong about the built-in yaml package in Python!
The linked issue did help, but it's an odd fix.

[subraman007]

@fschlz How did the attached issue fix the issue? It talks about upgrading PyYaml which will not help here