venv creation inconsistency with video

[gtmaskall]

Hi Mike,

Just starting to work through this after hearing about it on your podcast. The recent move of the requirements.txt file to the src directory creates a mismatch with the video. The student is left wondering whether to run the python -m venv command in snake_bnb or in the src subdirectory (where the requirements.txt is now). Well, I wondered.

Also I'm using an anaconda python3 dist, and I got

python3 -m venv .env --copies
Error: Command '['..../mongodb-quickstart-course/src/snake_bnb/.env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 127.

What worked for me was
python3 -m venv --without-pip .env --copies
This created the .env directory in snake_bnb alongside src. Hope this is as intended.

[mikeckennedy]

Hi,

That is funky. I have never seen that error. What version of Python 3 are you running? Is it possible you have an old one which does not have pip or are running on Linux with a limited install without pip?

In terms of location, I always do it in what will be the top level folder in PyCharm. That way it autodetects it and saves me a step. But this is not required. Just need to activate it via the CLI or register it in PyCharm and it can be anywhere.

Best,
Michael

[ianmonat]

I got the same error as @gtmaskall, but his fix worked for me as well

[mikeckennedy]

That is odd. @ianmonat what happens if you run `python3 -m pip list'? I'm wondering if pip is missing from that install for some reason (not the env but the original one used to create the env).

[ianmonat]

This is the output from python3 -m pip list

alabaster (0.7.10)
anaconda-client (1.6.3)
anaconda-navigator (1.6.2)
anaconda-project (0.6.0)
appnope (0.1.0)
appscript (1.0.1)
asn1crypto (0.22.0)
astroid (1.4.9)
astropy (1.3.2)
Babel (2.4.0)
backports.shutil-get-terminal-size (1.0.0)
beautifulsoup4 (4.6.0)
bitarray (0.8.1)
blaze (0.10.1)
bleach (1.5.0)
bokeh (0.12.5)
boto (2.46.1)
Bottleneck (1.2.1)
cffi (1.10.0)
chardet (3.0.3)
clarifai (2.0.33)
click (6.7)
click-plugins (1.0.3)
cligj (0.4.0)
cloudpickle (0.2.2)
clyent (1.2.2)
colorama (0.3.9)
conda (4.3.21)
configparser (3.5.0)
contextlib2 (0.5.5)
cryptography (1.8.1)
cycler (0.10.0)
Cython (0.25.2)
cytoolz (0.8.2)
dask (0.14.3)
datashape (0.5.4)
decorator (4.0.11)
distributed (1.16.3)
Django (1.11.5)
docutils (0.13.1)
EasyProcess (0.2.3)
emoji (0.4.5)
entrypoints (0.2.2)
enum34 (1.1.6)
et-xmlfile (1.0.1)
fastcache (1.0.2)
Fiona (1.7.9)
Flask (0.12.2)
Flask-Cors (3.0.2)
future (0.15.2)
geojsonio (0.0.3)
gevent (1.2.1)
github3.py (0.9.6)
greenlet (0.4.12)
h5py (2.7.0)
HeapDict (1.0.0)
html5lib (0.999)
idna (2.5)
imagesize (0.7.1)
ipykernel (4.6.1)
ipython (5.3.0)
ipython-genutils (0.2.0)
ipywidgets (6.0.0)
isort (4.2.5)
itsdangerous (0.24)
jdcal (1.3)
jedi (0.10.2)
Jinja2 (2.9.6)
jsonschema (2.5.1)
jupyter (1.0.0)
jupyter-client (5.0.1)
jupyter-console (5.1.0)
jupyter-core (4.3.0)
lazy-object-proxy (1.2.2)
llvmlite (0.18.0)
locket (0.2.0)
lxml (3.7.3)
MarkupSafe (0.23)
matplotlib (2.0.2)
mistune (0.7.4)
mongoengine (0.15.0)
mpmath (0.19)
msgpack-python (0.4.8)
multipledispatch (0.4.9)
munch (2.2.0)
navigator-updater (0.1.0)
nbconvert (5.1.1)
nbformat (4.3.0)
networkx (1.11)
nltk (3.2.3)
nose (1.3.7)
notebook (5.0.0)
numba (0.33.0)
numexpr (2.6.2)
numpy (1.12.1)
numpydoc (0.6.0)
odo (0.5.0)
olefile (0.44)
openpyxl (2.4.7)
packaging (16.8)
pandas (0.20.1)
pandocfilters (1.4.1)
partd (0.3.8)
pathlib2 (2.2.1)
patsy (0.4.1)
pep8 (1.7.0)
pexpect (4.2.1)
pickleshare (0.7.4)
Pillow (2.9.0)
pip (9.0.1)
ply (3.10)
prompt-toolkit (1.0.14)
psutil (5.2.2)
ptyprocess (0.5.1)
py (1.4.33)
pycosat (0.6.2)
pycparser (2.17)
pycrypto (2.6.1)
pycurl (7.43.0)
pyflakes (1.5.0)
Pygments (2.2.0)
pylint (1.6.4)
pymongo (3.6.0)
pyodbc (4.0.16)
pyOpenSSL (17.0.0)
pyparsing (2.1.4)
pyproj (1.9.5.1)
pytest (3.0.7)
python-dateutil (2.6.0)
pytz (2017.2)
PyVirtualDisplay (0.2.1)
PyWavelets (0.5.2)
PyYAML (3.12)
pyzmq (16.0.2)
QtAwesome (0.4.4)
qtconsole (4.3.0)
QtPy (1.2.1)
requests (2.13.0)
rope-py3k (0.9.4.post1)
schedule (0.5.0)
scikit-image (0.13.0)
scikit-learn (0.18.1)
scipy (0.19.0)
seaborn (0.7.1)
selenium (3.8.1)
setuptools (27.2.0)
Shapely (1.5.17.post1)
simplegeneric (0.8.1)
singledispatch (3.4.0.3)
six (1.10.0)
snowballstemmer (1.2.1)
sortedcollections (0.5.3)
sortedcontainers (1.5.7)
Sphinx (1.5.6)
spyder (3.1.4)
SQLAlchemy (1.1.9)
statsmodels (0.8.0)
sympy (1.0)
tables (3.3.0)
tblib (1.3.2)
terminado (0.6)
testpath (0.3)
toolz (0.8.2)
tornado (4.5.1)
tqdm (4.19.5)
traitlets (4.3.2)
unicodecsv (0.14.1)
uritemplate (3.0.0)
uritemplate.py (3.0.2)
wcwidth (0.1.7)
Werkzeug (0.12.2)
wheel (0.29.0)
widgetsnbextension (2.0.0)
wrapt (1.10.10)
xlrd (1.0.0)
XlsxWriter (0.9.6)
xlwings (0.10.4)
xlwt (1.2.0)
zict (0.1.2)

[mikeckennedy]

Hi guys. I don't think the copies is the issue. I think it's pip and a version mismatch. Have a look at this issue:

pypa/virtualenv#1059

In particular, near the bottom:

• Install 'python3.6-venv', and it should work.
• Wow, it didn't occur to me at all that there would be a version specific -venv package. Installing this did the trick.

Does that help?

[gtmaskall]

Hi Michael,

Thanks for your time! Following my nose I think it's less of a version mismatch and more of a distro mismatch. I have a data science background and anaconda is a popular distro for data science. I can do pandas etc but was wanting to develop my python with more developer stuff (such as mongodb). I think a key related issue is actually this one. Particularly, note ensurepip wasn't bundled with the anaconda dist. A user there seemed to update his python from 3.6.2 to 3.6.3 (screenshot) and then had ensurepip. I'm going to hazard that anaconda added ensurepip for python 3.6.3.

I did
conda update python
and went from 3.6.2 to 3.6.4. My venv now works as expected:

guy@jaka ~/projects/talkpythontraining/mongodb-quickstart-course/src/snake_bnb $ ls -a
.  ..  src
guy@jaka ~/projects/talkpythontraining/mongodb-quickstart-course/src/snake_bnb $ python3 -m venv .env --copies
guy@jaka ~/projects/talkpythontraining/mongodb-quickstart-course/src/snake_bnb $ 

Now I hope I can continue with your mongodb course! It was interesting to see the distro bundling discussion earlier in that thread. I'm sure such subtleties of python distros and virtual environments (venv vs conda) etc will bite me yet more times ahead! Perhaps a topic for an excellent podcast. ;-)

Cheers,
Guy

[ianmonat]

I got this error msg when trying to install python3.6-venv, am I doing something wrong?

Macintosh:snake_bnb ianmonat$ pip install python3.6-venv
Collecting python3.6-venv
  Could not find a version that satisfies the requirement python3.6-venv (from versions: )
No matching distribution found for python3.6-venv

[gtmaskall]

Ian,
What python distro are you using? I think this issue should be associated specifically with anaconda as I'm pretty happy it's the issue that ensurepip wasn't bundled between python ?? and 3.6.2 and that this issue can be closed with the solution being to update to python >= 3.6.3, thus
conda update python

The fact that you say the --without-pip option worked for you, makes me think it's the ensurepip issue and that you're probably using anaconda? If you are and still <= 3.6.2 then run the above update and that should resolve it. If not, then perhaps worth opening a new issue?

Guy

[ianmonat]

Macintosh:~ ianmonat$ python
Python 3.6.1 |Anaconda 4.4.0 (x86_64)| (default, May 11 2017, 13:04:09) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Does this answer your question? If I'm hearing you right, if I upgrade my version of python it should solve? Is this done on the cmd line or new download at python.org?

[gtmaskall]

Okay, cool. Yes, so you're running an Anaconda distro and your python is version 3.6.1 so if you run
conda update python
from the terminal, it should upgrade to version 3.6.4 which has ensurepip included. If you check starting with a clean clone of the mongodb course repo, you should be able to run the
python3 -m venv .env --copies
command as described and get no error.

Regards,
Guy

[ianmonat]

Thanks Guy - i ran the update...

Python 3.6.2 |Anaconda custom (x86_64)| (default, Jul 20 2017, 13:14:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

...deleted the project, re-cloned it, and got back to the point in the course where I had to stop previously, the point in which I actually opened a new issue about the missing image....

/Users/ianmonat/Desktop/OneDrive/Python_Scripts/mongodb-quickstart-course/src/snake_bnb/.env/bin/python /Users/ianmonat/Desktop/OneDrive/Python_Scripts/mongodb-quickstart-course/src/snake_bnb/src/program.py
dyld: Library not loaded: @rpath/libpython3.6m.dylib
  Referenced from: /Users/ianmonat/Desktop/OneDrive/Python_Scripts/mongodb-quickstart-course/src/snake_bnb/.env/bin/python
  Reason: image not found

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

....not sure this course is for me, will try a MongoDB University beginner course that opened up today and and maybe try again later...

[gtmaskall]

Hi Ian,
Your python is still < 3.6.3, although incremented from your previous 3.6.1. I don't know why it didn't update to 3.6.4. Perhaps your conda wants updating, try
conda update conda
followed by
conda update python
again. Does that get you to 3.6.4? (I'm on Linux but I'm assuming the Mac distribution will also be up to 3.6.4).

I'm going to close this issue now, but if you get up to python 3.6.4, get the specific error I started with above (despite python version 3.6.4), then do re-open this. I think it's most helpful to others to keep this issue pertinent to the lack of ensurepip in conda's python prior to 3.6.3.

I see your other issue and I'll drop you a reply in there with a couple of links that may be useful to you. I think your issue revolves around the discrepancy between "stock" python and anaconda's distribution; there are frequent comments that pip and virtual environments are not necessarily equivalent/compatible between the two. I assure you I am no expert. Part of my motivation is to expand from "just" using python for data science analysis and pick up more "development" such as accessing mongodb and using virtual environments. Thus, despite warnings/caveats from others on the Internet about subtle discrepancies when mixing conda and pip, with python 3.6.4 I'm now able to follow on with Michael's course, which is really the main criterion here rather than making a robust piece of software that doesn't break with some change to pip or conda!

I'd tentatively suggest you spend a little time familiarising yourself with your python install (e.g. I think compare Mac homebrew python with conda and pick which you want to use?) as I think that will stand you in good stead for any courses.

Regards,
Guy

[mikeckennedy]

Thanks for all the help and ideas @gtmaskall. @ianmonat if you are still having trouble, consider just using python3 -m venv with the straight up Python 3. The dependencies should install OK.

Best,
Michael