Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dropping executable bit on library files in virtualenv==20.0.14 #1752

Closed
rcramblit opened this issue Mar 26, 2020 · 7 comments · Fixed by #1754
Closed

Dropping executable bit on library files in virtualenv==20.0.14 #1752

rcramblit opened this issue Mar 26, 2020 · 7 comments · Fixed by #1754
Assignees
@gaborbernat
Labels
bug

Comments

@rcramblit
Copy link

Issue

virtualenv==20.0.14 appears to be losing the executable bit on some library files when installing pyspark. I have experienced this behavior on both Ubuntu and MacOS, with python3.6.9 and 3.8.1.

Virtualenv created with virtualenv==2.0.13:

(pyspark_13) ross: ~/WorkZone/VirtualEnvironments/pyspark_13
$ pip list
Package    Version
---------- -------
pip        20.0.2
py4j       0.10.7
pyspark    2.4.4
setuptools 46.0.0
wheel      0.34.2
(pyspark_13) ross: ~/WorkZone/VirtualEnvironments/pyspark_13
$ ll lib/python3.8/site-packages/pyspark/bin/
total 224
-rwxr-xr-x  1 ross  staff   1.1K Mar 26 09:07 beeline
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:07 beeline.cmd
-rwxr-xr-x  1 ross  staff   5.3K Mar 26 09:07 docker-image-tool.sh
-rwxr-xr-x  1 ross  staff   1.9K Mar 26 09:07 find-spark-home
-rw-r--r--  1 ross  staff   2.6K Mar 26 09:07 find-spark-home.cmd
-rw-r--r--  1 ross  staff   1.8K Mar 26 09:07 load-spark-env.cmd
-rw-r--r--  1 ross  staff   2.0K Mar 26 09:07 load-spark-env.sh
-rwxr-xr-x  1 ross  staff   2.9K Mar 26 09:07 pyspark
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 pyspark.cmd
-rw-r--r--  1 ross  staff   1.5K Mar 26 09:07 pyspark2.cmd
-rwxr-xr-x  1 ross  staff   1.0K Mar 26 09:07 run-example
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:07 run-example.cmd
-rwxr-xr-x  1 ross  staff   3.1K Mar 26 09:07 spark-class
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:07 spark-class.cmd
-rw-r--r--  1 ross  staff   2.8K Mar 26 09:07 spark-class2.cmd
-rwxr-xr-x  1 ross  staff   3.0K Mar 26 09:07 spark-shell
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:07 spark-shell.cmd
-rw-r--r--  1 ross  staff   1.8K Mar 26 09:07 spark-shell2.cmd
-rwxr-xr-x  1 ross  staff   1.0K Mar 26 09:07 spark-sql
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 spark-sql.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 spark-sql2.cmd
-rwxr-xr-x  1 ross  staff   1.0K Mar 26 09:07 spark-submit
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:07 spark-submit.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 spark-submit2.cmd
-rwxr-xr-x  1 ross  staff   1.0K Mar 26 09:07 sparkR
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 sparkR.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:07 sparkR2.cmd

Virtualenv create with virtualenv==2.0.14:

(pyspark_14) ross: ~/WorkZone/VirtualEnvironments/pyspark_14
$ pip list
Package    Version
---------- -------
pip        20.0.2
py4j       0.10.7
pyspark    2.4.4
setuptools 46.1.1
wheel      0.34.2
(pyspark_14) ross: ~/WorkZone/VirtualEnvironments/pyspark_14
$ ll lib/python3.8/site-packages/pyspark/bin/
total 224
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 beeline
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:13 beeline.cmd
-rw-r--r--  1 ross  staff   5.3K Mar 26 09:13 docker-image-tool.sh
-rw-r--r--  1 ross  staff   1.9K Mar 26 09:13 find-spark-home
-rw-r--r--  1 ross  staff   2.6K Mar 26 09:13 find-spark-home.cmd
-rw-r--r--  1 ross  staff   1.8K Mar 26 09:13 load-spark-env.cmd
-rw-r--r--  1 ross  staff   2.0K Mar 26 09:13 load-spark-env.sh
-rw-r--r--  1 ross  staff   2.9K Mar 26 09:13 pyspark
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 pyspark.cmd
-rw-r--r--  1 ross  staff   1.5K Mar 26 09:13 pyspark2.cmd
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:13 run-example
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:13 run-example.cmd
-rw-r--r--  1 ross  staff   3.1K Mar 26 09:13 spark-class
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:13 spark-class.cmd
-rw-r--r--  1 ross  staff   2.8K Mar 26 09:13 spark-class2.cmd
-rw-r--r--  1 ross  staff   3.0K Mar 26 09:13 spark-shell
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:13 spark-shell.cmd
-rw-r--r--  1 ross  staff   1.8K Mar 26 09:13 spark-shell2.cmd
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:13 spark-sql
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 spark-sql.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 spark-sql2.cmd
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:13 spark-submit
-rw-r--r--  1 ross  staff   1.2K Mar 26 09:13 spark-submit.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 spark-submit2.cmd
-rw-r--r--  1 ross  staff   1.0K Mar 26 09:13 sparkR
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 sparkR.cmd
-rw-r--r--  1 ross  staff   1.1K Mar 26 09:13 sparkR2.cmd

Environment

Provide at least:

  • OS: MacOS 10.14.6 (also observed on Ubuntu 16.0.4)
  • pip list of the host python where virtualenv is installed:

virtualenv==20.0.13:

$ python3.8 -m pip list
Package    Version
---------- -------
appdirs    1.4.3
distlib    0.3.0
filelock   3.0.12
pip        19.3.1
setuptools 42.0.2
six        1.14.0
virtualenv 20.0.13
wheel      0.33.6

virtualenv==20.0.14:

(virtualenv_14) ross: ~/WorkZone/VirtualEnvironments
$ pip list
Package    Version
---------- -------
appdirs    1.4.3
distlib    0.3.0
filelock   3.0.12
pip        20.0.2
setuptools 45.2.0
six        1.14.0
virtualenv 20.0.14
wheel      0.34.2

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

virtualenv==20.0.13:

ross: ~/WorkZone/VirtualEnvironments
$ virtualenv pyspark_13 -vvv --with-traceback
56 setup logging to NOTSET [DEBUG report:43]
88 find interpreter for spec PythonSpec(path=/usr/local/opt/python@3.8/bin/python3.8) [INFO builtin:44]
88 proposed PythonInfo(spec=CPython3.8.1.final.0-64, exe=/usr/local/opt/python@3.8/bin/python3.8, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
88 accepted PythonInfo(spec=CPython3.8.1.final.0-64, exe=/usr/local/opt/python@3.8/bin/python3.8, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
96 filesystem is not case-sensitive [DEBUG info:28]
253 create virtual environment via CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_13, clear=False, global=False) [INFO session:52]
253 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin [DEBUG _sync:21]
254 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages [DEBUG _sync:21]
255 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/pyvenv.cfg [DEBUG pyenv_cfg:34]
255 	home = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255 	implementation = CPython [DEBUG pyenv_cfg:38]
255 	version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
255 	virtualenv = 20.0.13 [DEBUG pyenv_cfg:38]
255 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
255 	base-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255 	base-exec-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
255 	base-executable = /usr/local/opt/python@3.8/bin/python3.8 [DEBUG pyenv_cfg:38]
256 symlink /usr/local/opt/python@3.8/bin/python3.8 to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python [DEBUG _sync:40]
258 create /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:78]
259 create virtualenv import hook file /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:81]
259 ============================== target debug ============================== [DEBUG session:54]
259 debug via /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python /usr/local/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
259 {
  "sys": {
    "executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python",
    "_base_executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/python",
    "prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13",
    "base_prefix": "/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8",
    "real_prefix": null,
    "exec_prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13",
    "base_exec_prefix": "/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8",
    "path": [
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
      "/Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]",
  "os": "<module 'os' from '/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
  "json": "<module 'json' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
315 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
316 Attempting to acquire lock 4418023584 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:270]
316 Lock 4418023584 acquired on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:274]
319 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
320 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/setuptools-46.0.0-py3-none-any.whl [DEBUG acquire:51]
320 get bundled wheel /usr/local/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
325 install wheel from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
326 install setuptools from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/setuptools-46.0.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
326 install pip from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
327 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools-46.0.0.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools-46.0.0.dist-info [DEBUG _sync:48]
327 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
327 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
328 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel [DEBUG _sync:48]
328 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip [DEBUG _sync:48]
342 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools-46.0.0.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools-46.0.0.virtualenv [DEBUG _sync:48]
344 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/easy_install.py to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:48]
346 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/setuptools to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/setuptools [DEBUG _sync:48]
354 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
368 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel3 to 755 [INFO util:566]
369 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel-3.8 to 755 [INFO util:566]
370 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/wheel to 755 [INFO util:566]
370 generated console scripts wheel-3.8 wheel3 wheel [DEBUG base:53]
457 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.0.0-py3-none-any/pkg_resources to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:48]
481 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install-3.8 to 755 [INFO util:566]
482 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install to 755 [INFO util:566]
482 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/easy_install3 to 755 [INFO util:566]
483 generated console scripts easy_install3 easy_install easy_install-3.8 [DEBUG base:53]
789 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/lib/python3.8/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip3.8 to 755 [INFO util:566]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip3 to 755 [INFO util:566]
799 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip to 755 [INFO util:566]
800 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/bin/pip-3.8 to 755 [INFO util:566]
800 generated console scripts pip-3.8 pip pip3.8 pip3 [DEBUG base:53]
800 Attempting to release lock 4418023584 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:315]
800 Lock 4418023584 released on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:318]
800 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
807 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_13/pyvenv.cfg [DEBUG pyenv_cfg:34]
807 	home = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807 	implementation = CPython [DEBUG pyenv_cfg:38]
807 	version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
807 	virtualenv = 20.0.13 [DEBUG pyenv_cfg:38]
807 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
807 	base-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807 	base-exec-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
807 	base-executable = /usr/local/opt/python@3.8/bin/python3.8 [DEBUG pyenv_cfg:38]
808 created virtual environment CPython3.8.1.final.0-64 in 753ms
  creator CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_13, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]

virtualenv==20.0.14:

(virtualenv_14) ross: ~/WorkZone/VirtualEnvironments
$ virtualenv pyspark_14 -vvv --with-traceback
92 setup logging to NOTSET [DEBUG report:43]
113 find interpreter for spec PythonSpec(path=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python) [INFO builtin:44]
114 discover exe for PythonInfo(spec=CPython3.8.1.final.0-64, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) in /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8 [DEBUG py_info:352]
115 filesystem is not case-sensitive [DEBUG info:28]
116 Attempting to acquire lock 4565512544 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [DEBUG filelock:270]
116 Lock 4565512544 acquired on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [INFO filelock:274]
117 get PythonInfo from /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.json for /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/bin/python3.8 [DEBUG cached_py_info:79]
117 Attempting to release lock 4565512544 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [DEBUG filelock:315]
117 Lock 4565512544 released on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/d02d962d4848e39588a949973e0d71f6f00b6517ee060386e3fea19a33a7b565.lock [INFO filelock:318]
118 Attempting to acquire lock 4565513456 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [DEBUG filelock:270]
118 Lock 4565513456 acquired on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [INFO filelock:274]
118 get PythonInfo from /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.json for /usr/local/opt/python@3.8/bin/python3.8 [DEBUG cached_py_info:79]
119 Attempting to release lock 4565513456 on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [DEBUG filelock:315]
119 Lock 4565513456 released on /Users/ross/Library/Application Support/virtualenv/py_info/20.0.14/1ee38ef4a5a9feb55287fd749643f13d043cb0a7addaab2a9c224cbe137c0062.lock [INFO filelock:318]
119 proposed PythonInfo(spec=CPython3.8.1.final.0-64, system=/usr/local/opt/python@3.8/bin/python3.8, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:50]
119 accepted PythonInfo(spec=CPython3.8.1.final.0-64, system=/usr/local/opt/python@3.8/bin/python3.8, exe=/Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/bin/python, platform=darwin, version='3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:52]
223 create virtual environment via CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_14, clear=False, global=False) [INFO session:52]
223 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin [DEBUG _sync:21]
223 create folder /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages [DEBUG _sync:21]
223 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/pyvenv.cfg [DEBUG pyenv_cfg:34]
223 	home = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
223 	implementation = CPython [DEBUG pyenv_cfg:38]
224 	version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
224 	virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
224 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
224 	base-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
224 	base-exec-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
224 	base-executable = /usr/local/opt/python@3.8/bin/python3.8 [DEBUG pyenv_cfg:38]
224 symlink /usr/local/opt/python@3.8/bin/python3.8 to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python [DEBUG _sync:40]
225 create virtualenv import hook file /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/_virtualenv.pth [DEBUG api:76]
226 create /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/_virtualenv.py [DEBUG api:79]
226 ============================== target debug ============================== [DEBUG session:54]
226 debug via /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
226 {
  "sys": {
    "executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python",
    "_base_executable": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/python",
    "prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14",
    "base_prefix": "/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8",
    "real_prefix": null,
    "exec_prefix": "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14",
    "base_exec_prefix": "/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8",
    "path": [
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python38.zip",
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8",
      "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload",
      "/Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.8.1 (default, Dec 27 2019, 18:05:45) \n[Clang 11.0.0 (clang-1100.0.33.16)]",
  "os": "<module 'os' from '/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py'>",
  "site": "<module 'site' from '/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/site.py'>",
  "datetime": "<module 'datetime' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/datetime.py'>",
  "math": "<module 'math' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/math.cpython-38-darwin.so'>",
  "json": "<module 'json' from '/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py'>"
} [DEBUG session:55]
275 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
276 Attempting to acquire lock 4570361376 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:270]
276 Lock 4570361376 acquired on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:274]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/setuptools-46.1.1-py3-none-any.whl [DEBUG acquire:51]
278 get bundled wheel /Users/ross/WorkZone/VirtualEnvironments/virtualenv_14/lib/python3.8/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
283 install wheel from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
284 install setuptools from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/setuptools-46.1.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
284 install pip from wheel /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
285 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
286 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/easy_install.py to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/easy_install.py [DEBUG _sync:48]
286 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
287 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel [DEBUG _sync:48]
287 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip [DEBUG _sync:48]
288 copy /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.virtualenv to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools-46.1.1.virtualenv [DEBUG _sync:48]
289 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools [DEBUG _sync:48]
306 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
316 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel-3.8 to 755 [INFO util:566]
317 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel to 755 [INFO util:566]
317 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/wheel3 to 755 [INFO util:566]
318 generated console scripts wheel3 wheel wheel-3.8 [DEBUG base:53]
380 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/pkg_resources to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pkg_resources [DEBUG _sync:48]
398 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/setuptools-46.1.1.dist-info [DEBUG _sync:48]
410 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install to 755 [INFO util:566]
411 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install-3.8 to 755 [INFO util:566]
411 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/easy_install3 to 755 [INFO util:566]
412 generated console scripts easy_install-3.8 easy_install3 easy_install [DEBUG base:53]
645 copy directory /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/lib/python3.8/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
653 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip-3.8 to 755 [INFO util:566]
654 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip3.8 to 755 [INFO util:566]
654 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip3 to 755 [INFO util:566]
655 changing mode of /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/bin/pip to 755 [INFO util:566]
655 generated console scripts pip3.8 pip3 pip-3.8 pip [DEBUG base:53]
655 Attempting to release lock 4570361376 on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [DEBUG filelock:315]
655 Lock 4570361376 released on /Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1/3.8/wheels.lock [INFO filelock:318]
655 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:64]
659 write /Users/ross/WorkZone/VirtualEnvironments/pyspark_14/pyvenv.cfg [DEBUG pyenv_cfg:34]
659 	home = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659 	implementation = CPython [DEBUG pyenv_cfg:38]
659 	version_info = 3.8.1.final.0 [DEBUG pyenv_cfg:38]
659 	virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
659 	include-system-site-packages = false [DEBUG pyenv_cfg:38]
659 	base-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659 	base-exec-prefix = /usr/local/Cellar/python@3.8/3.8.1/Frameworks/Python.framework/Versions/3.8 [DEBUG pyenv_cfg:38]
659 	base-executable = /usr/local/opt/python@3.8/bin/python3.8 [DEBUG pyenv_cfg:38]
660 created virtual environment CPython3.8.1.final.0-64 in 569ms
  creator CPython3Posix(dest=/Users/ross/WorkZone/VirtualEnvironments/pyspark_14, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/ross/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]
@rcramblit rcramblit added the bug label Mar 26, 2020
@gaborbernat gaborbernat self-assigned this Mar 26, 2020
@gaborbernat
Copy link
Contributor

A PR would be welcomed; we need to understand when and why this happens first.

@rcramblit
Copy link
Author

It appears the change might be in the setuptools version bump - looking at the diff between the versions I see this: pypa/setuptools@7843688#diff-c02383bc12d5e72a50c798501bba7984

@asottile
Copy link
Contributor

I believe this is due to setuptools >=46.1,<46.1.3 -- in particular pypa/setuptools#1424

it appears to be fixed here: pypa/setuptools#2046 (46.1.3)

@ianw ianw mentioned this issue Mar 26, 2020
Closed
@ianw
Copy link
Contributor

ianw commented Mar 26, 2020

Yes I agree it is setuptools 46.1.1 as embedded currently.

You can work around it by running virtualenv -p python3 --seeder pip --download and you will grab the latest from PyPI; things are working for me doing that and using 46.1.3

@gaborbernat
Copy link
Contributor

You don't need another seeder, just pass in the --download flag will suffice 👍 or the --setuptools 46.1.3 flag

@karelyatin
Copy link

Can we have good version of setuptools 46.1.3 bundled with virtualenv instead of 46.1.1 to avoid workarounds.

@gaborbernat gaborbernat mentioned this issue Mar 27, 2020
Merged
@gaborbernat
Copy link
Contributor

Released via https://pypi.org/project/virtualenv/20.0.15/

@pypa pypa locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@gaborbernat gaborbernat

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

upgrade embedded setuptools from 46.1.1 to 46.1.3 gaborbernat/virtualenv
5 participants
@ianw @gaborbernat @asottile @rcramblit @karelyatin