[BUG] dependency grpcio wheel build fails with Python 3.12 on Ubuntu 24.04

[dirkpetersen]

Ubuntu 24.04 was just released and nvflare fails to install with the default Python 3.12.

Looks a bit like this grpc/grpc#33063 but that one is already fixed. Note that running python3 -m pip install --break-system-packages grpcio==1.62.1 on Ubuntu 24.04 installs a wheel just fine.

Desktop (please complete the following information):

• OS: Ubuntu 24.04
• Python Version: 3.12
• NVFlare Version: 2.4

To Reproduce
python3 -m pip install nvflare

  src/python/grpcio/grpc/_cython/cygrpc.cpp:133891:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  133891 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: In function ‘grpc_call_error __Pyx_PyInt_As_grpc_call_error(PyObject*)’:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134070:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134070 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134125:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134125 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: In function ‘size_t __Pyx_PyInt_As_size_t(PyObject*)’:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134266:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134266 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134321:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134321 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: In function ‘unsigned int __Pyx_PyInt_As_unsigned_int(PyObject*)’:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134500:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134500 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp:134555:55: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  134555 |             const digit* digits = ((PyLongObject*)x)->ob_digit;
         |                                                       ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: In function ‘Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*)’:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:135947:47: error: ‘PyLongObject’ {aka ‘struct _longobject’} has no member named ‘ob_digit’
  135947 |     const digit* digits = ((PyLongObject*)b)->ob_digit;
         |                                               ^~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: In function ‘void __pyx_f_4grpc_7_cython_6cygrpc__unified_socket_write(int)’:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:60644:15: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
  60644 |   (void)(write(__pyx_v_fd, ((char *)"1"), 1));
        |         ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/python/grpcio/grpc/_cython/cygrpc.cpp: At global scope:
  src/python/grpcio/grpc/_cython/cygrpc.cpp:135081:1: warning: ‘void __Pyx_PyAsyncGen_Fini()’ defined but not used [-Wunused-function]
  135081 | __Pyx_PyAsyncGen_Fini(void)
         | ^~~~~~~~~~~~~~~~~~~~~
  creating tmp
  creating tmp/tmpo74d7tre
  x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/usr/include/python3.12 -c /tmp/tmpo74d7tre/a.c -o tmp/tmpo74d7tre/a.o
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
      self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
      _classic_spawn(self, command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
      spawn(cmd, dry_run=self.dry_run, **kwargs)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/spawn.py", line 70, in spawn
      raise DistutilsExecError(
  distutils.errors.DistutilsExecError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/src/python/grpcio/commands.py", line 280, in build_extensions
      build_ext.build_ext.build_extensions(self)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 249, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
                ^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
      self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
      raise CompileError(msg)
  distutils.errors.CompileError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/setup.py", line 535, in <module>
      setuptools.setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1233, in run_command
      super().run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/home/ubuntu/.local/lib/python3.12/site-packages/wheel/bdist_wheel.py", line 368, in run
      self.run_command("build")
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1233, in run_command
      super().run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1233, in run_command
      super().run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 88, in run
      _build_ext.run(self)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/src/python/grpcio/commands.py", line 284, in build_extensions
      raise CommandError(
  commands.CommandError: Failed `build_ext` step:
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
      self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
      _classic_spawn(self, command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
      spawn(cmd, dry_run=self.dry_run, **kwargs)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/spawn.py", line 70, in spawn
      raise DistutilsExecError(
  distutils.errors.DistutilsExecError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/tmp/pip-install-_e5hzm1j/grpcio_339b92de5f6346f595c43d3fd6a665b1/src/python/grpcio/commands.py", line 280, in build_extensions
      build_ext.build_ext.build_extensions(self)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 249, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
      objects = self.compiler.compile(
                ^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
      self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
      raise CompileError(msg)
  distutils.errors.CompileError: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for grpcio
Running setup.py clean for grpcio
Failed to build grpcio
ERROR: Could not build wheels for grpcio, which is required to install pyproject.toml-based projects

[chesterxgchen]

@dirkpetersen thanks for reporting, currently we only support python version up to 3.10, we will consider to expand the support 3.11 and/or 3.12 in the future releases.