Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breakage in --always-copy - fails with: ImportError: No module named time #565

Closed
ionelmc opened this issue Feb 12, 2014 · 26 comments
Closed

Comments

@ionelmc
Copy link

ionelmc commented Feb 12, 2014

On CentOS 6.5 (python2.7 from SCL):

[dev@centbox core]$ virtualenv-2.7 --always-copy .break
sys.path (patched logging) = ['/opt/rh/python27/root/usr/bin', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/wheel-0.22.0-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/pip-1.5.2-py2.7.egg', '/opt/rh/python27/root/usr/lib64/python27.zip', '/opt/rh/python27/root/usr/lib64/python2.7', '/opt/rh/python27/root/usr/lib64/python2.7/plat-linux2', '/opt/rh/python27/root/usr/lib64/python2.7/lib-tk', '/opt/rh/python27/root/usr/lib64/python2.7/lib-old', '/opt/rh/python27/root/usr/lib64/python2.7/lib-dynload', '/opt/rh/python27/root/usr/lib64/python2.7/site-packages', '/opt/rh/python27/root/usr/lib/python2.7/site-packages']
Cannot find file lib (bad symlink)
New python executable in .break/bin/python2.7
Also creating executable in .break/bin/python
Installing setuptools, pip...
  Complete output from command /home/dev/core/.break/bin/python2.7 -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv_support/pip-1.5.2-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv_support/pip-1.5.2-py2.py3-none-any.whl/pip/log.py", line 6, in <module>
  File "/opt/rh/python27/root/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "/opt/rh/python27/root/usr/bin/virtualenv-2.7", line 9, in <module>
    load_entry_point('virtualenv==1.11.2', 'console_scripts', 'virtualenv-2.7')()
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv.py", line 992, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv.py", line 960, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg/virtualenv.py", line 902, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/dev/core/.break/bin/python2.7 -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

Works fine without it:

[dev@centbox core]$ virtualenv-2.7 .works
sys.path (patched logging) = ['/opt/rh/python27/root/usr/bin', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/wheel-0.22.0-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/virtualenv-1.11.2-py2.7.egg', '/opt/rh/python27/root/usr/lib/python2.7/site-packages/pip-1.5.2-py2.7.egg', '/opt/rh/python27/root/usr/lib64/python27.zip', '/opt/rh/python27/root/usr/lib64/python2.7', '/opt/rh/python27/root/usr/lib64/python2.7/plat-linux2', '/opt/rh/python27/root/usr/lib64/python2.7/lib-tk', '/opt/rh/python27/root/usr/lib64/python2.7/lib-old', '/opt/rh/python27/root/usr/lib64/python2.7/lib-dynload', '/opt/rh/python27/root/usr/lib64/python2.7/site-packages', '/opt/rh/python27/root/usr/lib/python2.7/site-packages']
New python executable in .works/bin/python2.7
Also creating executable in .works/bin/python
Installing setuptools, pip...done.
[dev@centbox core]$
@asimihsan
Copy link

I'm hitting this issue too, but as a workaround you can add no-setuptools to not install pip in your virtualenv. This issue is related to installing pip/setuptools in the new virtualenv.

My version information:

  • Fedora 19 x86_64,
  • System install of Python 2.7.5
  • virtualenv 1.11.4
  • pip 1.5.4

Here is the full verbose output on failure:

[vagrant@localhost ~]$ virtualenv --always-copy --clear --verbose foobar
Do not need to delete foobar/lib/python2.7; already gone
Not deleting foobar/bin
Creating foobar/lib/python2.7
Cannot find file lib (bad symlink)
Symlinking Python bootstrap modules
  Copying to foobar/lib/python2.7/config
  Copying to foobar/lib/python2.7/lib-dynload
  Creating parent directories for foobar/lib64/python2.7
  Copying to foobar/lib64/python2.7/os.py
  Ignoring built-in bootstrap module: posix
  Copying to foobar/lib64/python2.7/posixpath.py
  Cannot import bootstrap module: nt
  Copying to foobar/lib64/python2.7/ntpath.py
  Copying to foobar/lib64/python2.7/genericpath.py
  Copying to foobar/lib64/python2.7/fnmatch.py
  Copying to foobar/lib64/python2.7/locale.py
  Copying to foobar/lib64/python2.7/encodings
  Copying to foobar/lib64/python2.7/codecs.py
  Copying to foobar/lib64/python2.7/stat.py
  Copying to foobar/lib64/python2.7/UserDict.py
  Creating parent directories for foobar/lib64/python2.7/lib-dynload
  Copying to foobar/lib64/python2.7/lib-dynload/readline.so
  Copying to foobar/lib64/python2.7/copy_reg.py
  Copying to foobar/lib64/python2.7/types.py
  Copying to foobar/lib64/python2.7/re.py
  Copying to foobar/lib64/python2.7/sre.py
  Copying to foobar/lib64/python2.7/sre_parse.py
  Copying to foobar/lib64/python2.7/sre_constants.py
  Copying to foobar/lib64/python2.7/sre_compile.py
  Copying to foobar/lib64/python2.7/lib-dynload/zlibmodule.so
  Copying to foobar/lib64/python2.7/warnings.py
  Copying to foobar/lib64/python2.7/linecache.py
  Copying to foobar/lib64/python2.7/_abcoll.py
  Copying to foobar/lib64/python2.7/abc.py
  Copying to foobar/lib64/python2.7/_weakrefset.py
Creating foobar/lib/python2.7/site-packages
Writing foobar/lib64/python2.7/site.py
Writing foobar/lib64/python2.7/orig-prefix.txt
Writing foobar/lib64/python2.7/no-global-site-packages.txt
Creating parent directories for foobar/include
Copying to foobar/include/python2.7
Creating foobar/bin
New python executable in foobar/bin/python
Changed mode of foobar/bin/python to 0775
Copying to foobar/bin/python2
Copying to foobar/bin/python2.7
Testing executable with foobar/bin/python -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
Got sys.prefix result: u'/home/vagrant/foobar'
Creating foobar/lib64/python2.7/distutils
Writing foobar/lib64/python2.7/distutils/__init__.py
Writing foobar/lib64/python2.7/distutils/distutils.cfg
Installing setuptools, pip...
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
    File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/log.py", line 6, in <module>
    File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
      import sys, os, time, cStringIO, traceback, warnings, weakref
  ImportError: No module named time
  Complete output from command /home/vagrant/foobar/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/log.py", line 6, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 824, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 992, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 960, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 902, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/vagrant/foobar/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

Here is the full verbose output on success:

[vagrant@localhost ~]$ virtualenv --always-copy --clear --verbose --no-setuptools foobar
Do not need to delete foobar/lib/python2.7; already gone
Not deleting foobar/bin
Creating foobar/lib/python2.7
Cannot find file lib (bad symlink)
Symlinking Python bootstrap modules
  Copying to foobar/lib/python2.7/config
  Copying to foobar/lib/python2.7/lib-dynload
  Creating parent directories for foobar/lib64/python2.7
  Copying to foobar/lib64/python2.7/os.py
  Ignoring built-in bootstrap module: posix
  Copying to foobar/lib64/python2.7/posixpath.py
  Cannot import bootstrap module: nt
  Copying to foobar/lib64/python2.7/ntpath.py
  Copying to foobar/lib64/python2.7/genericpath.py
  Copying to foobar/lib64/python2.7/fnmatch.py
  Copying to foobar/lib64/python2.7/locale.py
  Copying to foobar/lib64/python2.7/encodings
  Copying to foobar/lib64/python2.7/codecs.py
  Copying to foobar/lib64/python2.7/stat.py
  Copying to foobar/lib64/python2.7/UserDict.py
  Creating parent directories for foobar/lib64/python2.7/lib-dynload
  Copying to foobar/lib64/python2.7/lib-dynload/readline.so
  Copying to foobar/lib64/python2.7/copy_reg.py
  Copying to foobar/lib64/python2.7/types.py
  Copying to foobar/lib64/python2.7/re.py
  Copying to foobar/lib64/python2.7/sre.py
  Copying to foobar/lib64/python2.7/sre_parse.py
  Copying to foobar/lib64/python2.7/sre_constants.py
  Copying to foobar/lib64/python2.7/sre_compile.py
  Copying to foobar/lib64/python2.7/lib-dynload/zlibmodule.so
  Copying to foobar/lib64/python2.7/warnings.py
  Copying to foobar/lib64/python2.7/linecache.py
  Copying to foobar/lib64/python2.7/_abcoll.py
  Copying to foobar/lib64/python2.7/abc.py
  Copying to foobar/lib64/python2.7/_weakrefset.py
Creating foobar/lib/python2.7/site-packages
Writing foobar/lib64/python2.7/site.py
Writing foobar/lib64/python2.7/orig-prefix.txt
Writing foobar/lib64/python2.7/no-global-site-packages.txt
Creating parent directories for foobar/include
Copying to foobar/include/python2.7
Creating foobar/bin
New python executable in foobar/bin/python
Changed mode of foobar/bin/python to 0775
Copying to foobar/bin/python2
Copying to foobar/bin/python2.7
Testing executable with foobar/bin/python -c "import sys;out=sys.stdout;getattr(out, "buffer", out).write(sys.prefix.encode("utf-8"))"
Got sys.prefix result: u'/home/vagrant/foobar'
Creating foobar/lib64/python2.7/distutils
Writing foobar/lib64/python2.7/distutils/__init__.py
Writing foobar/lib64/python2.7/distutils/distutils.cfg
Writing foobar/bin/activate
Writing foobar/bin/activate.fish
Writing foobar/bin/activate_this.py
Writing foobar/bin/activate.csh

@asimihsan
Copy link

In both the failing and successful cases the first error line is:

Cannot find file lib (bad symlink)

This log isn't useful because I don't have the fullpath to the file it can't find. I hence hacked virtualenv.py to print out os.path.abspath for this file and got:

Cannot find file /home/vagrant/lib (bad symlink)

It's looking for a lib folder in the current working directory.

@dstufft dstufft mentioned this issue Jan 5, 2015
5 tasks
@lhw365
Copy link

lhw365 commented Jan 19, 2015

the same here on Centos7 with stock python 2.7

$ virtualenv --always-copy /opt/
New python executable in /opt/bin/python
Installing setuptools, pip...
  Complete output from command /opt/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-6.0.6-py2.py3-none-any.whl/pip/__init__.py", line 4, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "/usr/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 825, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 993, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 961, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 903, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /opt/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1

@famousgarkin
Copy link

Same here. Using --always-copy on CentOS 6.5, system Python 2.6 ends up with same ImportError: No module named time as mentioned.

@lhw365
Copy link

lhw365 commented Feb 20, 2015

the solution is to forget about the --always-copy option on EL based distros

This was referenced May 14, 2015
@bwilliams42
Copy link

there seems to be a typo in the fix_lib64 function that is currently running. I correct this and it works as expected. go to line:

   1579         if symlink:                              
   1580             os.symlink('lib', lib64_link)      
   1581         else:                                    
   1582             copyfile('lib', lib64_link)        

replace 'lib' with lib_dir

  1579         if symlink:                              
   1580             os.symlink(lib_dir, lib64_link)      
   1581         else:                                    
   1582             copyfile(lib_dir, lib64_link)        

Late, but hope this helps

@jdelgad
Copy link

jdelgad commented Jul 10, 2015

I'm getting the same problem here as well using CentOS 7.1 with flags --no-setup-tools and --always-copy. I'd love for this to be fixed since I'm packaging up the virtualenv and shipping them to customers (that are also on CentOS7+ so it's not a major risk).

@selurvedu
Copy link

Same here on Gentoo x86_64 with --always-copy.

@bwilliams42
Copy link

@jdelgad and @selurvedu did you try what i recommended above?

@selurvedu
Copy link

@ikiini Yes, your patch solves the problem.

@jdelgad
Copy link

jdelgad commented Aug 23, 2015

@ikiini It worked! Thank you so much. Now to either get this into RHEL 7 or into the main branch.

@Kuckkuck
Copy link

+1

@bwilliams42
Copy link

great. Sorry for the delayed response..but you're welcome @selurvedu and @jdelgad.

@Ivoz
Copy link

Ivoz commented Oct 21, 2015

Only problem seems to be @ikiini 's patch would seem to regress back on #338 which is when it was actually changed from lib_dir to 'lib'

@lakshmi-kannan
Copy link

Problem still persists with CentOS 7 and python 2.7

[root@st2w-master-el7-5696a8a082 ~]# virtualenv no_copy
New python executable in no_copy/bin/python
Installing setuptools, pip, wheel...done.
[root@st2w-master-el7-5696a8a082 ~]# virtualenv --always-copy yes_copy
Cannot find file lib (bad symlink)
New python executable in yes_copy/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /root/yes_copy/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-7.1.2-py2.py3-none-any.whl/pip/__init__.py", line 4, in <module>
  File "/usr/lib64/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref
ImportError: No module named time
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 832, in main
    symlink=options.symlink)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 1004, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 969, in install_wheel
    'PIP_NO_INDEX': '1'
  File "/usr/lib/python2.7/site-packages/virtualenv.py", line 910, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /root/yes_copy/bin/python -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip wheel failed with error code 1
[root@st2w-master-el7-5696a8a082 ~]# virtualenv --version
13.1.2
[root@st2w-master-el7-5696a8a082 ~]# ls -rlth yes_copy/
total 0
drwxr-xr-x. 3 root root 22 Jan 13 22:04 lib
drwxr-xr-x. 3 root root 22 Jan 13 22:04 lib64
drwxr-xr-x. 3 root root 22 Jan 13 22:04 include
drwxr-xr-x. 2 root root 49 Jan 13 22:04 bin
[root@st2w-master-el7-5696a8a082 ~]#

pjfontillas added a commit to pjfontillas/dutchman that referenced this issue Jan 22, 2016
Ran into a virtualization error that took a while to debug:
gratipay/gratipay.com#2327
pypa/virtualenv#565
Ultimate solution: http://stackoverflow.com/questions/24640819/protocol-error-setting-up-virtualenvironment-through-vagrant-on-ubuntu
Run console as admin, shared mounts permissions issue.
More evidence of issues on Windows: bash script cannot access vagrant shared mount, even when run as root.
@ckochenower
Copy link

@bwilliams42 - Your change indeed fixed the problem for symlink-ing logic. However, things are fundamentally broken when not using --always-copy. Note that copyfile has a "symlink" kwarg that defaults to True. The following logic is effectively, if symlink, call os.symlink, else call copyfile w/o the "symlink" kwarg which results in copyfile defaulting symlink=True and a symlink is created by copyfile. Not useful at all. A symlink is the result no matter what.

Shouldn't the patch be something like the following?

   1579         if symlink:                              
   1580             os.symlink(lib_dir, lib64_link)      
   1581         else:                                    
   1582             copyfile(lib_dir, lib64_link, symlink=symlink)

@jdelgad
Copy link

jdelgad commented Feb 28, 2016

I had an issue whereby I did this and I had a symlink to /usr/lib64. When RH updated some of the openssl libraries this was broken (I was using ansible) with an error stating something like openssl_meth function. I fixed it by doing a deep copy of /usr/lib64. Not a complete fix @bwilliams42 but pointed me in the right direction.

@bialix
Copy link

bialix commented Oct 18, 2016

Problem still persists on Centos 7.2. This ticket has been reported 2.5 years ago. It seems it's very serious problem. How can I help to fix it?

@pfmoore
Copy link
Member

pfmoore commented Oct 18, 2016

Someone needs to provide a working fix. There's one fix that's been proposed here, but it apparently has a couple of problems. I'd suggest starting there.

@bialix
Copy link

bialix commented Oct 18, 2016

BTW, I'm trying to build and pack my python appliction into virtual env on special VM under Vagrant with help of fabric, so it's pretty easy to reproduce this bug with Vagrant + centos box + simple fabric script.

I don't have inner knowledge about how virtualenv is supposed to work, so I need some guidance.

@pfmoore
Copy link
Member

pfmoore commented Oct 18, 2016

I can't help there sorry, as I'm on Windows, and the whole question of symlinking vs copying (or lib64 directories, which seem involved in this) isn't relevant there.

@kika
Copy link

kika commented May 30, 2017

Interesting that (I'm on Amazon Linux) when I do virtualenv --always-copy app I get this error (Cannot find file lib (bad symlink)) but when I do mkdir app; cd app; virtualenv --always-copy . I do not. And it seems I understand why - when the app is created by the virtualenv it has lib64 as a directory, but when I create the virtualenv in the current directory it becomes a link to lib.

@ederag
Copy link

ederag commented Jan 27, 2018

virtualenv.py > def path_locations starts with this comment:

XXX: We'd use distutils.sysconfig.get_python_inc/lib but its
prefix arg is broken: http://bugs.python.org/issue3386

This issue has been solved about ten years ago. Maybe reconsider using it ?
Just giving another angle; writing a reliable fix for all platforms is by no means an easy task.

@roaming-zen
Copy link

@kika Thank you so much.

@stale
Copy link

stale bot commented Apr 11, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

@stale stale bot added the wontfix label Apr 11, 2019
@gaborbernat
Copy link
Contributor

This is becoming obsolete with #1366 now in full swing. We'll reopen if someone can still replicate with that.

@pypa pypa locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests