Python 3 support

[hadim]

Any plan to support Python 3 for mayavi ?

Python going to release Python 3.4 soon. I think Python 3 is now very mature and main third party library (numpy, scipy, pandas, matplotlib, etc) support it now.

I am also slowly moving all my project to Python 3. It's pretty simple task with 2to3 app for "small" project. You can sometime get some conflict with old std libs or issues with unicode/string stream but nothing bad...

I guess it will be a bit more complicated to move mayavi and others related projects but it shouldn't be something too hard to do in a mid term time.

I don't want to talk for others but I think many people are waiting mayavi to move their projects to Python 3.

"Don't wait until Python 4 is released please..."

:-)

Cheers

[dpinte]

@hadim The plans for Mayavi on Python 3 are linked to VTK being available on Python 3. The roadmap to get there is unclear. See http://www.vtk.org/Wiki/VTK/Python_Wrapping_FAQ.

[inducer]

👍

[gauteh]

👍

[rutsky]

👍

[jenshnielsen]

I don't think a 👍 here is gone do much good. You should rather try to convince Kitware to support python3 in VTK. Without that there is not any chance for a python3 version of Mayavi.

[GaelVaroquaux]

I don't think a 👍 here is gone do much good. You should rather try to
convince Kitware to support python3 in VTK. Without that there is not any
chance for a python3 version of Mayavi.

Thank you for stressing that!

[rutsky]

Maybe you can provide link to proper bugreport in VTK about porting it to Python 3, to redirect users saying +1 there?

[hadim]

For the record I opened a ticket on VTK bug tracker : http://www.vtk.org/Bug/view.php?id=15554

[jabooth]

Great news! David Gobbi (the maintainer of the VTK Python bindings) has responded on that bug ticket and stated that he's already landed merge requests to prepare the bindings for Py3 support, and he's concretely working on Py3 support starting this weekend:
http://www.vtk.org/Bug/view.php?id=15554#c34787

I completely understand that this is just the start of the process and it may be some time before Python 3 support lands in a release of VTK, but I wonder if there is anything Mayavi could actively be doing to prepare itself for Python 3? I take it there will be changes needed to the Mayavi codebase itself to support Python 3, is it worth starting a discussion around what this work would be?

[itziakos]

Thanks @jabooth,

As far as I can tell the full picture for Mayavi moving to Python 3 depends on the main dependencies supporting python 3. The current status is:

• python vtk
• traits
• traitsui
• pyface
• envisage

Python 3 VTK bindings is the main part and the fact that there is work taking place is good news, but without the bindings available on a stable release of VTK it would be very difficult to work and test on a python 3 friendly Mayavi.

Furthermore, Mayavi itself depends heavily on traits, traitsui and pyface which need to also support python 3. While there is work-in-progress in these libraries support is not there yet.

Nevertheless, what we can do at this moment, (and @dmsurti started already) is to fix pending bugs improve testing coverage and simplify the Mayavi codebase so that when the dependencies are available in python 3 the effort for moving Mayavi to Python 3 will be less.

[toddrjen]

You can also do things like switch to "from future import
print_function, division", get rid of any relative imports, and switching
everything over to six.

[prabhuramachandran]

I think doing anything right now is a waste of time. Mayavi does not do too much which cannot be fixed in a day of work, the bigger issue is VTK supporting Python 3 and as soon as that is available we will make the necessary change. Without VTK, we will not be able to test. I am aware of David Gobbi's recent work but I am going to wait until it is officially supported and merged into VTK.

[mabl]

Looks like the python-py3k branch was just merged: https://gitlab.kitware.com/vtk/vtk/merge_requests/478

Does anyone know the status of tvtk? If i could finally transition to py3k this year that would so great!

[mabl]

$ python
Python 3.4.3 (default, Mar 25 2015, 17:13:50) 
[GCC 4.9.2 20150304 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
>>>

Yeehaa!

[mabl]

Is there any progress on this? It looks like Python 3 support will be in the next minor release vtk 6.3.1.

[jenshnielsen]

See #250

[stefanoborini]

pull request #250 is merged. Closing.

[solarjoe]

Just some quick postive feedback
I am using WinPython-64bit-3.4.4.1 with Mayavi 4.4.4 and it works fine. Thanks to all devs!

1.) VTK-7.0.0-cp34-cp34m-win_amd64.whl (http://www.lfd.uci.edu/~gohlke/pythonlibs/#vtk)
2.) pip install mayavi

[solarjoe]

@IVIUPPET,
does the workaround mentioned here
#250 (comment)
fix the problem?

[IVIUPPET]

@solarjoe

HAven't tried it, but I came back here to post my working results for others to follow, coming from very limited Python background, but experience with Matlab and C#.

I was getting an unsupported platform error that I commented about then promptly deleted, because I was trying to use your link (for Python 3.4) with Python 3.5
But I appreciate your reply. Comment with my setup to follow.

[IVIUPPET]

I installed/am using Anaconda for Python 3.5. Version 3.19.1. I used pip 8.1.1. (not that either of these matter much). 64 bit whenver possible.

I then downloaded:

1. VTK-7.0.0-cp35-cp35m-win_amd64.whl
   From: http://www.lfd.uci.edu/~gohlke/pythonlibs/#vtk
   python 3.5 64 bit version of VTK, a critical dependency for Mayavi (Thanks solarjoe!)

cd to downloadDir:
pip install VTK-7.0.0-cp35-cp35m-win_amd64.whl

Took me a bit to solve absence of vcvarsall.bat, which was the next issue I ran into once I found this thread to get VTK working with python 3.5. Tried adding stuff to different environment variables, installing old versions of Visual Studio. Didn't work. Ended up just getting Visual Studio Community 2015, which contains C++ compiler for Python 3.5. I know, it's overkill, but I spent hours doing the other stuff, so for me this was the path of least resistance (I'm an EE). During install, be sure to do "custom" and then under "programming languages", check anything that has to do with Python or C++.

2. pip install mayavi

It worked. After a day of googling I finally have 3D graphing that seem to match or exceed Matlab! my Matlab to Python transition is complete :D

[indranilsinharoy]

As I was reading @IVIUPPET 's comments above, I remembered a very recent (April 11th) blog post from Microsoft named How to deal with the pain of "unable to find vcvarsall.bat". Although I am not certain, I thought that post might help.

[jabooth]

@dreme: we had a user with a similar issue see if their resolution helps: menpo/menpo3d#16

The resulting instructions in our README might be easier to follow:
https://github.com/menpo/menpo3d#visualizing-3d-objects

Key part:

If you are running Windows and recieve this error, try:

set QT_API=pyqt
set ETS_TOOLKIT=qt4

@jenshnielsen I tried to automate this so that users (of Menpo at least) wouldn't have to worry about it, but I couldn't get it to work: menpo/menpo3d#22

I'm going to raise a separate issue to try and discuss improving the situation here.

[dreme]

You should also be able to set the env variable from python before importing mayavi. http://stackoverflow.com/questions/5971312/how-to-set-environment-variables-in-python

Bingo! That did the trick. Thx @jenshnielsen =:-)

I much prefer this in-script approach compared to doing a system-wide OS setting of the QT_API environmental variable, as I may still want to do pyqt5 development in my root Anaconda environment.

So, in summary for any poor shmuck like me who is struggling to get mayavi running on anaconda python 3.5, here is my suggested approach distilled from the helpful advice in preceding comments:

1. Create a new env (oldqt) which is downgraded to pyqt4 by running the following command in the anaconda root env terminal:
   conda create -n oldqt python=3 pyqt=4

2. Switch to the newly created env:
   activate oldqt for Windows OS
   source activate oldqt for Mac/linux OS

3. install mayavi using the menpo package installer:
   conda install -c menpo mayavi

4. Use the oldqt env to run your python mayavi scripts, and include the snippet below at the top of these scripts to ensure that a valid QT API for mayavi is specified:

import os
os.environ["QT_API"] = "pyqt"

[rgoalcast]

VTK-7.0.0-cp34-cp34m-win_amd64.whl is not avalable at http://www.lfd.uci.edu/~gohlke/pythonlibs/#vtk, from where I can get VTK-7.0.0-cp34-cp34m-win_amd64.whl.

[solarjoe]

It was available a week ago, I downloaded it. It is still on my office computer I could send you a link on
Monday (also for the matching Mayavi version).

Why don't you use the newer Mayavi and VTK?

[rgoalcast]

Please send link for VTK-7.0.0-cp34-cp34m-win_amd64.whl. Thank you.

[solarjoe]

Link below should be valid about 30 days.

VTK-7.0.0-cp34-cp34m-win_amd64.whl

http://ge.tt/8QvtGkl2

[nevion]

so... approximately where is this at? I been keeping my eye on this since inception and it's another one of those projects that just doesn't seem to happen with.

[mabl]

@nevion How do you mean? This is a closed issue, and I have mayavi working on Python 3.6 on Windows as well as Linux.

[nevion]

@mabl It wan't really clear from the issue trail that this was the case, and things are often left in an unclear state in scientific software with something like qt support laggin. Between this and the anaconda distribution only supporting 2.7, I thought this was still the case. Any idea why anaconda still only supports 2.7?

conda install mayavi
Fetching package metadata .........
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - mayavi -> apptools 4.2.0 -> python 2.7* -> openssl 1.0.1*
  - python 3.6*
Use "conda info <package>" to see the dependencies for each package.

[nthallen]

Just installed Anaconda3 and tried to install mayavi. Got:

(C:\ProgramData\Anaconda3) C:\Users\nort.ARP>conda install mayavi
Fetching package metadata ...........
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:

• mayavi -> apptools 4.2.0 -> python 2.7*
• python 3.6*
  Use "conda info " to see the dependencies for each package.

[jonathanrocher]

Did you look at/try the 4-point instructions above from @dreme ?

[scivision]

Specifically from @dreme

1. Create a new env (oldqt) which is downgraded to pyqt4 by running the following command in the anaconda root env terminal:

        conda create -n oldqt python=3.5 pyqt=4
   

2. Switch to the newly created env:

        activate oldqt #for Windows OS
        source activate #oldqt for Mac/linux OS
   

3. install mayavi using the menpo package installer:

        conda install -c menpo mayavi
   

4. Use the oldqt env to run your python mayavi scripts, and include the snippet below at the top of these scripts to ensure that a valid QT API for mayavi is specified:

        import os
        os.environ["QT_API"] = "pyqt"
   

[nthallen]

@jonathanrocher No, fair enough. I was perhaps misreading the assertion that this was solved and just worked. Yes, it installed following the recipe.

[raphael-attie]

I just tried @dreme 4-point instructions and it failed with python 3.6. I had to force 3.5:
conda create -n oldqt python=3.5 pyqt=4 and that worked.

[scivision]

yes that's correct, Mayavi 4.5 requires Python 3.5 due to QT4

[larsoner]

So I can get mayavi working on Qt5 and Python 3.6 + VTK7.1 using e.g. conda install python=3.6.1 vtk>=7 -c clinicalgraphics, and using some recent commit numbers from pyface, traits, and traitsui. This seems to work on OSX, Windows, and Linux according to our CIs.

However, on OSX machines with HiDPI support, only 1/4 of the screen area is used:

Is this something that can be fixed at the Mayavi level?

It seems like it was maybe meant to be part of VTK7.1:

https://public.kitware.com/pipermail/vtk-developers/2016-November/034456.html

But before looking deeper I figured I'd ask in case others had ideas.

[larsoner]

One option (sad but would fill the window) is to wait for VTK8 to come out, then use QVTKOpenGLWidget, which you can use to turn HiDPI off:

https://www.vtk.org/doc/nightly/html/classQVTKOpenGLWidget.html#details

Or maybe HiDPI will "just work" when using this widget.

I've tried the environment variables listed here but they do not fix it.

[mpanighel]

Hi @larsoner, are there some news now that vtk 8.1 is available?

Installing vtk 8.1.0 conda install -c conda-forge vtk, python is downgraded to 3.6.2 (and matplotlib to 2.1.0 between others).
I installed pyface 5.1.0, traits 4.6.0, traitsui 5.1.0, conda install -c conda-forge pyface traits traitsui
I have qt 5.6.2.

Installing mayavi 4.5.0 from conda-forge conda install -c conda-forge mayavi it complains about dependencies pyqt 4.*, python 3.6* and vtk 7.1.*.

Any suggestion to have it working with the current packages or, alternatively, exactly how did you made it work with Python3.6, Qt5 and VTK7.1? (Still there is the solution by @dreme for Python3.5+Qt4) Thank you.

[larsoner]

We have been using this environment file for CI for about 6 months on Linux, OSX, and Windows with good success (other than the HiDPI stuff):

https://github.com/mne-tools/mne-python/blob/master/environment.yml

It has a bunch of stuff that you don't need. The critical components for Mayavi were getting VTK7 from clinicalgraphics because they had a 3.6 version, and pinning to some commits of pyface, traits, and traitsui from GitHub in lieu of using releases as the Qt5-supporting versions have not been released yet.

If VTK8 is available on conda-forge, it could probably safely be swapped in, but I haven't tried it yet. I'll give it a shot at some point and see if it fixes the HiDPI stuff. If you beat me to it, let me know how it goes!

[prabhuramachandran]

@larsoner, @mpanighel -- this is an issue with VTK 7.x (the window being only quarter the size) and works fine with VTK 8.x. Stay tuned for an easier way to install VTK in a few days.

[prabhuramachandran]

Just FYI, I have pushed wheels for VTK 8.1.0 on PyPI, see here: http://prabhuramachandran.blogspot.in/2018/01/vtk-810-wheels-for-all-platforms-on-pypi.html

[larsoner]

Awesome @prabhuramachandran !

[mpanighel]

That is great @prabhuramachandran ! Thanks!

In these days I followed your latest fixes and tried to reproduce your setup and test mayavi on python3.6/pyqt5 on Linux, but with no luck (surely for my very basic python knowledge), most probably I just missing some obvious package. I thought to post more info in #595 to be tidier.