scikit-build-core encountering temporary directory permission error?

[mattrobin]

Hi! I'm trying to run the minimal example from the scikit-build-core "Getting started" documentation, but I'm running in an error during the build. I originally found the error in a larger project, but when I couldn't figure it out there, I attempted the minimal configuration from the documentation, and it was still occurring there as well. Specifically, when I run the pip install . for the example code, I encounter:

(example_dev_env) [mrobin@gs66-bower]~/scikit_build_core_minimal_example% pip install .
Processing /local/data/bowerssd1/mrobin/scikit_build_core_minimal_example
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: example
  Building wheel for example (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for example (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [42 lines of output]
      *** scikit-build-core 0.7.1 using CMake 3.20.2 (wheel)
      2024-01-08 23:48:58,217 - scikit_build_core - WARNING - No license files found, set wheel.license-files to [] to suppress this warning
      *** Configuring CMake...
      Traceback (most recent call last):
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/build/__init__.py", line 31, in build_wheel
          return _build_wheel_impl(
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/build/wheel.py", line 289, in _build_wheel_impl
          builder.configure(
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/builder/builder.py", line 146, in configure
          local_def = set_environment_for_gen(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/builder/generator.py", line 110, in set_environment_for_gen
          ninja = best_program(
                  ^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/program_search.py", line 131, in best_program
          for program in programs:
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/program_search.py", line 98, in get_ninja_programs
          result = Run().capture(ninja_path, "--version")
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/_shutil.py", line 43, in capture
          return self._run(args, capture=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiq6m26j/overlay/lib/python3.11/site-packages/scikit_build_core/_shutil.py", line 71, in _run
          return subprocess.run(
                 ^^^^^^^^^^^^^^^
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/subprocess.py", line 548, in run
          with Popen(*popenargs, **kwargs) as process:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/subprocess.py", line 1026, in __init__
          self._execute_child(args, executable, preexec_fn, close_fds,
        File "/local/data/bowerssd1/mrobin/miniforge3/envs/example_dev_env/lib/python3.11/subprocess.py", line 1950, in _execute_child
          raise child_exception_type(errno_num, err_msg, err_filename)
      PermissionError: [Errno 13] Permission denied: '/tmp/pip-build-env-uiq6m26j/normal/lib/python3.11/site-packages/ninja/data/bin/ninja'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for example
Failed to build example
ERROR: Could not build wheels for example, which is required to install pyproject.toml-based projects

Notably, I'm only getting this on a Linux server, but not on my local macOS machine. I've tried completely removing my install of conda (miniforge) and performing a fresh install. I've tried multiple environments with different versions of Python. I tried moving all the temporary directory variables (TMPDIR, TMP, and TEMP) to a location not in the /tmp directory, though none of these seemed to change the process. It seems to be occurring during the internal environment build step from scikit-build-core, as the versions of cmake, etc that I install in the environment do not set the versions that are used during the build, so I assume the scikit-build-core is creating another environment and installing its own versions in there for the build. I've tried requiring that scikit-build-core is set to use the latest versions of cmake and ninja as the minimal versions.

If anyone has a suggestion as to what might be going wrong, it would be greatly appreciated. Thank you for your time!

[thewtex]

Hi @mattrobin,

One idea is to check whether /tmp/ was mounted without exec capabilities.

[henryiii]

If you install ninja (and cmake, if it's too old) yourself, you'll also get to skip the usage of the pypi ones; they are only requested if a system one is not found. But I'd guess Matt is probably right, sounds like you can't execute anything in the virtual environment.

I wonder if there's an envvar to control the location of pip's isolated environment?

[mattrobin]

@thewtex, Indeed, it appears the /tmp directory on the server is mounted with noexec. Notably, I do not have permissions to change this though.

@henryiii, According to this issue, it sounds like pip should respect the TMPDIR specified by the user. As noted above, I have tried setting TMPDIR, TMP, and TEMP, to another location, but the build still uses /tmp. Does scikit-build-core do anything to ignore or override the temporary directory settings? As for the ninja version, I already have the latest ninja version installed in the environment that initiates the build, but it still seems to separately install ninja.

Thank you for your time.

[thewtex]

@mattrobin could you please test this patch and see if it addresses your use case? scikit-build/scikit-build-core#626

[mattrobin]

@thewtex, the patch (as submitted in version 0.8.1) seems to resolve the issue. Thank you much for your time and the time of the other developers!

[golmschenk]

@thewtex, it seems the issue still occurs for dependencies that are not CMake or ninja. For example, I have a project with fypp as a requirement for the build system, and the same issue occurs for that now.

[golmschenk]

I opened this as a separate issue.