failure to upgrade pynq in virtual python environment

[xlla]

I am follow the instruction in section Python Source Code to upgrade my pynq from v3.0.0 to latest, but it failed due to the virtual python environment.

round 1, it can't find python module pynqutils inside sudo

ModuleNotFoundError: No module named 'pynqutils'

sudo pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 836.7 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pynqmetadata>=0.0.1 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pynq) (0.3.1)
Requirement already satisfied: pynqutils>=0.0.1 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pynq) (0.1.0)
Building wheels for collected packages: pynq

Building wheel for pynq (pyproject.toml) ... error

error: subprocess-exited-with-error
× Building wheel for pynq (pyproject.toml) did not run successfully.

│ exit code: 1

╰─> [2854 lines of output]
....

thousands copying

....
  Traceback (most recent call last):
    File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
      self.run_setup()
    File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 267, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 431, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
      return run_commands(dist)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
      dist.run_commands()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
      cmd_obj.run()
    File "setup.py", line 367, in run
      copy_notebooks()
    File "setup.py", line 326, in copy_notebooks
      copy_overlay_notebooks(notebooks_dir, board)
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

account xilinx can import that module

xilinx@pynq:~$ python
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pynqutils.setup_utils import download_overlays
>>> quit()

sudo cannot import that module

xilinx@pynq:~$ sudo python3
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pynqutils.setup_utils import download_overlays
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'pynqutils'
>>> quit()

round 2, execute as normal account, failure again

ERROR: Will not install to the user site because it will lack sys.path precedence to pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages

xilinx@pynq:~$ pip3 --proxy=http://10.0.0.17:10809 install --upgrade --upgrade-strategy only-if-needed pynq
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 886.9 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pydantic->pynqmetadata>=0.0.1->pynq) (4.3.0)
ERROR: Will not install to the user site because it will lack sys.path precedence to pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages

round 3, use "sudo -E" option to keep current environment variables to sudo

xilinx@pynq:~$ sudo python3 -c "from pynqutils.setup_utils import download_overlays"
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:~$ sudo -E python3 -c "from pynqutils.setup_utils import download_overlays"
xilinx@pynq:~$ 

ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:~$ sudo -E pip3  install --upgrade --upgrade-strategy only-if-needed pynq
WARNING: The directory '/home/xilinx/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 5.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pynq
  Building wheel for pynq (pyproject.toml) ... error
  error: subprocess-exited-with-error
× Building wheel for pynq (pyproject.toml) did not run successfully.

│ exit code: 1

╰─> [2854 lines of output]

....

thousands copying

....
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 4, use "sudo -E -H" option to keep current environment variables and change the $Home environment variable, it remove the warnings in "round 3", but still failure.

ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:/home$ sudo -E -H  /usr/local/share/pynq-venv/bin/pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
....
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 5, use "sudo -H -E env PATH=$PATH " option to keep more environment variables, keep failure

ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:/home$ sudo -H -E env PATH=$PATH pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
....
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 6, use 'sudo sh -c ". /usr/local/share/pynq-venv/bin/activate ;" ' to activate python virtual env direct inside sudo, just failure

ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:/home$ sudo sh -c ". /usr/local/share/pynq-venv/bin/activate ; pip3 install --upgrade --upgrade-strategy only-if-needed pynq"
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
....
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 7, add "source /etc/profile.d/pynq_venv.sh" to root's .bashrc, execute command by root, failure ~ failure !!!

ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:/home$ sudo vi /root/.bashrc
xilinx@pynq:/home$ sudo -i
(pynq-venv) root@pynq:~# which python
/usr/local/share/pynq-venv/bin/python
(pynq-venv) root@pynq:~# cd /home/xilinx
(pynq-venv) root@pynq:/home/xilinx# pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
....
    File "setup.py", line 202, in copy_overlay_notebooks
      from pynqutils.setup_utils import download_overlays
  ModuleNotFoundError: No module named 'pynqutils'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for pynq

Failed to build pynq

ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

Because slow arm speed and slow SD card write speed, I spend hours to struggle with this issue, it seems in some files, it will invoke a subprocess and execute command using default python instead of pyqn_venv one.