Following extension tutorial instructions, when run "pip install -ve ." encounter prepare_metadata_for_build_editable issue

[gitist]

Description

Following the instructions in this page: https://jupyterlab.readthedocs.io/en/latest/extension/extension_tutorial.html#extension-tutorial, when as instructed I run the command: pip install -ve . in my_first_extension directory, I will get:
INFO:hatch_jupyter_builder.utils:> /tmp/pip-build-env-pcnfq6v7/overlay/bin/jlpm install

➤ YN0000: ┌ Resolution step
➤ YN0001: │ RequestError
at ClientRequest. (/tmp/pip-build-env-pcnfq6v7/overlay/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:195:14340)
at Object.onceWrapper (node:events:633:26)
at ClientRequest.emit (node:events:530:35)
at o.emit (/tmp/pip-build-env-pcnfq6v7/overlay/lib/python3.12/site-packages/jupyterlab/staging/yarn.js:190:90286)
at TLSSocket.socketErrorListener (node:_http_client:500:9)
at TLSSocket.emit (node:events:518:28)
at emitErrorNT (node:internal/streams/destroy:169:8)
at emitErrorCloseNT (node:internal/streams/destroy:128:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)AggregateError [ETIMEDOUT]:
at internalConnectMultiple (node:net:1116:18)
at internalConnectMultiple (node:net:1184:5)
at Timeout.internalConnectMultipleTimeout (node:net:1710:5)
at listOnTimeout (node:internal/timers:575:11)
at process.processTimers (node:internal/timers:514:7)
➤ YN0000: └ Completed in 19s 452ms
➤ YN0000: Failed with errors in 19s 455ms
Traceback (most recent call last):
File "/home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 167, in prepare_metadata_for_build_editable
hook = backend.prepare_metadata_for_build_editable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
main()
File "/home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 176, in prepare_metadata_for_build_editable
whl_basename = build_hook(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-pcnfq6v7/overlay/lib/python3.12/site-packages/hatchling/build.py", line 83, in build_editable
return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-pcnfq6v7/overlay/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 147, in build
build_hook.initialize(version, build_data)
File "/tmp/pip-build-env-pcnfq6v7/normal/lib/python3.12/site-packages/hatch_jupyter_builder/plugin.py", line 94, in initialize
raise e
File "/tmp/pip-build-env-pcnfq6v7/normal/lib/python3.12/site-packages/hatch_jupyter_builder/plugin.py", line 89, in initialize
build_func(self.target_name, version, **build_kwargs)
File "/tmp/pip-build-env-pcnfq6v7/normal/lib/python3.12/site-packages/hatch_jupyter_builder/utils.py", line 115, in npm_builder
run([*npm_cmd, "install"], cwd=str(abs_path))
File "/tmp/pip-build-env-pcnfq6v7/normal/lib/python3.12/site-packages/hatch_jupyter_builder/utils.py", line 231, in run
return subprocess.check_call(cmd, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/pip-build-env-pcnfq6v7/overlay/bin/jlpm', 'install']' returned non-zero exit status 1.
error: subprocess-exited-with-error

× Preparing editable metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/ss/miniconda3/envs/jupyterlab-ext/bin/python3.12 /home/ss/miniconda3/envs/jupyterlab-ext/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_editable /tmp/tmpg8hk813b
cwd: /home/ss/my_first_extension
Preparing editable metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Reproduce

Follow the instructions in the Extension Tutorial until you get to:
pip install -ve .

Expected behavior

I should be able to follow the instructions easily

Context

• Operating System and version: Ubuntu 20.04

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[gitist]

I have tried:

1. Changing hatchling version in pyproject.toml to 1.7 and 1.8. None helped.
2. Running "jlpm cache clean" and removing jupyterlab_apod directory. That didn't help either.
3. Creating vanilla venv instead of conda environment. No use!

Greatly appreciate the community's feedback asap!

[krassowski]

Are you trying to follow jupyterlab or jupyter-ai tutorial? If it is jupyterlab (which it sounds like based on mention of jupyterlab_apod) let me know and I will transfer the issue to the jupyterlab repository.

Ignore prepare_metadata_for_build_editable this is a red herring. The error you are getting is a RequestError on TLSSocket.socketErrorListener. Do you have a proxy configured for yarn? Anything unusual in network configuration?

[gitist]

I am behind a proxy server:

❯ env | grep proxy
HTTP_PROXY=http://proxy-.................................
http_proxy=http://proxy-...............................
HTTPS_PROXY=http://proxy-................................
https_proxy=http://proxy-.................................
(base)

As mentioned just followed the tutorial on https://jupyterlab.readthedocs.io/en/stable/extension/extension_tutorial.html
There is nothing there about yarn.

[dnoliver]

@krassowski I think it's a problem with jupyterlab tutorial. I tried this same steps behind a corporate proxy, and run into the same problem:

(this is windows with minicoda, and export proxy vars with set HTTP_PROXY=<proxy> set HTTPS_PROXY=<proxy>)

While running this command: pip install -ve ., I eventually run into this problem:

  Installing collected packages: trove-classifiers, pluggy, pathspec, packaging, editables, hatchling, hatch-jupyter-builder
  Successfully installed editables-0.5 hatch-jupyter-builder-0.9.1 hatchling-1.24.2 packaging-24.0 pathspec-0.12.1 pluggy-1.5.0 trove-classifiers-2024.4.10
  Installing backend dependencies ... done
  Running command Preparing editable metadata (pyproject.toml)
  INFO:hatch_jupyter_builder.utils:Running jupyter-builder
  INFO:hatch_jupyter_builder.utils:Building with hatch_jupyter_builder.npm_builder
  INFO:hatch_jupyter_builder.utils:With kwargs: {'build_cmd': 'install:extension', 'npm': ['jlpm'], 'source_dir': 'src', 'build_dir': 'myextension/labextension'}
  INFO:hatch_jupyter_builder.utils:Installing build dependencies with npm.  This may take a while...
  INFO:hatch_jupyter_builder.utils:> C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\overlay\Scripts\jlpm.EXE install
  ➤ YN0000: ┌ Resolution step
  ➤ YN0001: │ RequestError
      at ClientRequest.<anonymous> (C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\overlay\Lib\site-packages\jupyterlab\staging\yarn.js:195:14340)
      at Object.onceWrapper (node:events:633:26)
      at ClientRequest.emit (node:events:530:35)
      at o.emit (C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\overlay\Lib\site-packages\jupyterlab\staging\yarn.js:190:90286)
      at TLSSocket.socketErrorListener (node:_http_client:500:9)
      at TLSSocket.emit (node:events:518:28)
      at emitErrorNT (node:internal/streams/destroy:169:8)
      at emitErrorCloseNT (node:internal/streams/destroy:128:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)AggregateError [ETIMEDOUT]:
      at internalConnectMultiple (node:net:1116:18)
      at afterConnectMultiple (node:net:1683:7)
  ➤ YN0000: └ Completed in 1m 44s
  ➤ YN0000: Failed with errors in 1m 44s
  Traceback (most recent call last):
    File "C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 167, in prepare_metadata_for_build_editable
      hook = backend.prepare_metadata_for_build_editable
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 176, in prepare_metadata_for_build_editable
      whl_basename = build_hook(metadata_directory, config_settings)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\overlay\Lib\site-packages\hatchling\build.py", line 83, in build_editable
      return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\overlay\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build
      build_hook.initialize(version, build_data)
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\normal\Lib\site-packages\hatch_jupyter_builder\plugin.py", line 94, in initialize
      raise e
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\normal\Lib\site-packages\hatch_jupyter_builder\plugin.py", line 89, in initialize
      build_func(self.target_name, version, **build_kwargs)
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\normal\Lib\site-packages\hatch_jupyter_builder\utils.py", line 115, in npm_builder
      run([*npm_cmd, "install"], cwd=str(abs_path))
    File "C:\Users\dnoliver\AppData\Local\Temp\pip-build-env-cp9xom5p\normal\Lib\site-packages\hatch_jupyter_builder\utils.py", line 231, in run
      return subprocess.check_call(cmd, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['C:\\Users\\dnoliver\\AppData\\Local\\Temp\\pip-build-env-cp9xom5p\\overlay\\Scripts\\jlpm.EXE', 'install']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\python.exe' 'C:\Users\dnoliver\AppData\Local\miniconda3\envs\jupyterlab-ext\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' prepare_metadata_for_build_editable 'C:\Users\dnoliver\AppData\Local\Temp\tmp_wivnzvf'
  cwd: C:\Users\dnoliver\GitHub\jlabext\my_first_extension
  Preparing editable metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

feels like something is not respecting the proxy env vars. Could be Yarn (the internet says that it doesn't like proxy via env, and needs proxy via settings).

Workaround: switch out of the VPN/Network environment that requires using a web proxy :)