Jupyter Notebook kernel dies: Problems with py5 in M3 MacOs 14.4.1

[ivanlen]

Hi, there, I just follow all the steps and check similar github issues but I wasn't able to fix my problem.

• Macbook M3
• MacOs 14.4.1
• Jupyter notebook 7.1.3

I am trying to use py5 in a jupyter notebook.

My top cell runs

%gui osx

Then I just import py5 and the kernel dies:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000018779dc20, pid=4507, tid=259
#
# JRE version: OpenJDK Runtime Environment Homebrew (21.0.2) (build 21.0.2)
# Java VM: OpenJDK 64-Bit Server VM Homebrew (21.0.2, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
# Problematic frame:
# C  [libobjc.A.dylib+0x9c20]  objc_msgSend+0x20

Running py5_tools I get

In [1]: import py5_tools

In [2]: py5_tools.get_jvm_debug_info()
Out[2]:
{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home',
 'jvm version': (0, 0, 0),
 'default jvm path': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

Also since I am using Java 21 I installed

python -c "import jdk; print('Java installed to', jdk.install('21'))"

Happy to provide any other info.
Maybe related to #452 #316

[ivanlen]

Some extra info, import py5 from ipyhon (without using a jupyter notebook) works

In [3]: import py5
   ...:
   ...: def setup():
   ...:     py5.size(640, 360)
   ...:     py5.background(255)
   ...:
   ...: def draw():
   ...:     py5.fill(py5.random(255), py5.random(255), py5.random(255))
   ...:     py5.ellipse(py5.mouse_x, py5.mouse_y, 50, 50)
   ...:
   ...: py5.run_sketch()

Importing py5 on macOS but the necessary Jupyter macOS event loop has not been activated. I'll activate it for you, but next time, execute `%gui osx` before importing this library.

No crashes so far.

[hx2A]

Hello, @ivanlen , thank you for opening this issue.

I'm guessing this is going to be a challenging issue to address. The M3 computers are new and perhaps there is some new MacOS curveball to deal with.

Since you are on 14.4.1, you don't have this problem.

Here are a few things I will ask you to try that will help gather information.

1. What happens when you skip %gui osx and go right to import py5 in a Jupyter Notebook?
2. What if you try this in jupyter console? This looks very similar to the ipython REPL but internally it is different.
3. Can you try saving the example Sketch to a file and running it from the terminal with python example.py?
4. For any place where py5 works, can you check the output of py5_tools.get_jvm_debug_info()? It could be using a different JVM than the one that gets used from the Jupyter Notebook.
5. Can you try installing Java 17 and setting JAVA_HOME to point to that? It could be there is something odd about Java 21 on MacOS.

[ivanlen]

Thanks for the questions, let me reply inline.

Hello, @ivanlen , thank you for opening this issue.

I'm guessing this is going to be a challenging issue to address. The M3 computers are new and perhaps there is some new MacOS curveball to deal with.

Since you are on 14.4.1, you don't have this problem.
Yes, I upgraded just to make sure that it wasn't my problem. Also, since it is working in ipython but not in the notebook I suspected that java is working fine, but who knows....

Here are a few things I will ask you to try that will help gather information.

1. What happens when you skip %gui osx and go right to import py5 in a Jupyter Notebook?

Just tried it, still crashes. I receive the meesage Importing py5 on macOS but the necessary Jupyter macOS event loop has not been activated. I'll activate it for you, but next time, execute %gui osxbefore importing this library. and then kernel dies.

2. What if you try this in jupyter console? This looks very similar to the ipython REPL but internally it is different.

I get an infinite spam of the error

2024-05-13 08:57:18.193 python[58727:315773] *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:653

3. Can you try saving the example Sketch to a file and running it from the terminal with python example.py?

Yes, this works! I have tried this with ipython and it was working fine, also by running python test.py it is working fine.

4. For any place where py5 works, can you check the output of py5_tools.get_jvm_debug_info()? It could be using a different JVM than the one that gets used from the Jupyter Notebook.
   I have already tried this, sorry for not mentioning it before.

In jupyter

{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home',
 'jvm version': (0, 0, 0),
 'default jvm path': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

In ipython or python (that both are working)

{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home', 'jvm version': (0, 0, 0), 'default jvm path': '/opt/homebrew/Cellar/openjdk/21.0.2/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

5. Can you try installing Java 17 and setting JAVA_HOME to point to that? It could be there is something odd about Java 21 on MacOS.
   Done!
   Also I installed python -c "import jdk; print('Java installed to', jdk.install('17'))" just to make sure that all versions are aligned.

Now running

import py5_tools
py5_tools.get_jvm_debug_info()

{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home',
 'jvm version': (0, 0, 0),
 'default jvm path': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

However things are not working. I get the endless spam of Assertion failure errors

2024-05-13 09:57:06.123 python[84384:438408] *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:653

I am going to try to reinstall all the venv again to see if I have any luck (I am not very confident).

So to wrap up. both in ipyhon and with python py5 is working, however it is not working with jupyter gui or jupyter console. This happens with both, version 17 and 21 of java.

[hx2A]

So to wrap up. both in ipyhon and with python py5 is working, however it is not working with jupyter gui or jupyter console. This happens with both, version 17 and 21 of java.

Ok, there's a bit of a pattern emerging here. Here are some more things to try:

1. Run jupyter qtconsole. I predict this will fail just like the other Jupyter options.
2. What is the output of jupyter --version? It could be there are some weird change to the latest Jupyter packages that is tripping up py5. On my Linux machine in front of me, I get:

$ jupyter --version
Selected Jupyter core packages...
IPython          : 8.12.2
ipykernel        : 6.26.0
ipywidgets       : 8.1.1
jupyter_client   : 8.6.0
jupyter_core     : 5.5.0
jupyter_server   : 2.10.1
jupyterlab       : 4.0.9
nbclient         : 0.7.4
nbconvert        : 7.11.0
nbformat         : 5.9.2
notebook         : 7.0.4
qtconsole        : 5.5.1
traitlets        : 5.13.0

3. Install the py5 Jupyter kernel and try starting a notebook with this kernel. I predict the notebook will crash immediately.
4. Try Juypter Lab instead of Jupyter Notebook. This shouldn't make a difference though.
5. Also, for any place where py5 works, can you check the output of py5_tools.get_jvm_debug_info() after importing py5? It should at least fill in the version number.

[hx2A]

Some more things to try in Jupyter notebooks:

import jpype

jpype.startJVM()

print(jpype.isJVMStarted())

If this fails, then this is a problem that is separate from py5. You might be experiencing the same thing discussed in this thread.

If this works, then we'll have to see if the Jars py5 adds to the classpath before calling jpype.startJVM() are tripping this up. py5 also runs a tiny bit of code on import py5 to check if you have a headless JVM or not. Outsiide of that, it is not doing much and it is not clear to me why this is crashing.

Also, you didn't say which version of Python you are using? I don't think it matters but it might inform my thinking here.

[ivanlen]

So to wrap up. both in ipyhon and with python py5 is working, however it is not working with jupyter gui or jupyter console. This happens with both, version 17 and 21 of java.

Ok, there's a bit of a pattern emerging here. Here are some more things to try:

1. Run jupyter qtconsole. I predict this will fail just like the other Jupyter options.

jupyter qtconsole there are no errors, the console shows up. However when I import py5 in the qtconsole I have the same error as before.

2. What is the output of jupyter --version? It could be there are some weird change to the latest Jupyter packages that is tripping up py5. On my Linux machine in front of me, I get:

$ jupyter --version
Selected Jupyter core packages...
IPython          : 8.12.2
ipykernel        : 6.26.0
ipywidgets       : 8.1.1
jupyter_client   : 8.6.0
jupyter_core     : 5.5.0
jupyter_server   : 2.10.1
jupyterlab       : 4.0.9
nbclient         : 0.7.4
nbconvert        : 7.11.0
nbformat         : 5.9.2
notebook         : 7.0.4
qtconsole        : 5.5.1
traitlets        : 5.13.0

This is the output for me

Selected Jupyter core packages...
IPython          : 8.18.1
ipykernel        : 6.29.3
ipywidgets       : 8.1.2
jupyter_client   : 8.6.1
jupyter_core     : 5.7.2
jupyter_server   : 2.14.0
jupyterlab       : 4.1.8
nbclient         : 0.10.0
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.1.3
qtconsole        : 5.5.1
traitlets        : 5.14.3

If all the other things don't work I can try to reinstall the whole venv and also downgrading some packages (but I am going to try your other suggested points before).

3. Install the py5 Jupyter kernel and try starting a notebook with this kernel. I predict the notebook will crash immediately.

I am going to do this in a while. I am using jupyter for another thing and I don't want to break everything now (just in case)

4. Try Juypter Lab instead of Jupyter Notebook. This shouldn't make a difference though.

Yes, crashing as well...

5. Also, for any place where py5 works, can you check the output of py5_tools.get_jvm_debug_info() after importing py5? It should at least fill in the version number.

What do you mean here the version number, can you show me your py5_tools.get_jvm_debug_info() so I know what to expect?

[ivanlen]

Some more things to try in Jupyter notebooks:

import jpype

jpype.startJVM()

print(jpype.isJVMStarted())

If this fails, then this is a problem that is separate from py5. You might be experiencing the same thing discussed in this thread.

This is working, jpype.isJVMStarted() gives me True in the notebook.

If this works, then we'll have to see if the Jars py5 adds to the classpath before calling jpype.startJVM() are tripping this up. py5 also runs a tiny bit of code on import py5 to check if you have a headless JVM or not. Outsiide of that, it is not doing much and it is not clear to me why this is crashing.

Also, you didn't say which version of Python you are using? I don't think it matters but it might inform my thinking here.

Yess, you are right, sorry about this, I am using Python 3.9.18

Thanks for all the ideas. I hope we can fix it :)

[hx2A]

What do you mean here the version number, can you show me your py5_tools.get_jvm_debug_info() so I know what to expect?

Here is what I get. The jvm_version numbers are not (0, 0, 0).

>>> py5_tools.get_jvm_debug_info()
{'JAVA_HOME environment variable': '/usr/lib/jvm/java-17-openjdk',
 'jvm version': (17, 0, 9),
 'default jvm path': '/usr/lib/jvm/java-17-openjdk-17.0.9.0.9-3.fc39.x86_64/lib/server/libjvm.so'}

[hx2A]

Thanks for all the ideas. I hope we can fix it :)

Yes, I hope we can!

You can get the list of Jars added to the classpath with this:

jars = py5_tools.get_classpath().split(':')

Get the output of that from IPython or someplace where py5 works.

Then add that same list to the classpath with this:

import jpype

for jar in jars:
    jpype.addClassPath(jar)

jpype.startJVM()
print(jpype.isJVMStarted())

If this fails, see if you can isolate which Jar is tripping it up.

[ivanlen]

import py5_tools
py5_tools.get_jvm_debug_info()

Well, I have some news, I just created a new env. Reinstalled everything from scratch with Java17 activated (instead of 21) at the moment of the installation.

Now I am getting this from iPython

{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home',
 'jvm version': (17, 0, 10),
 'default jvm path': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

So on this front we are moving forward, however I am still getting the error. I am going to do what you have proposed in your other message.

[ivanlen]

jars = py5_tools.get_classpath().split(':')

I think we have something here (or the absence of something XD )

In [4]: import py5_tools

In [5]: py5_tools.get_jvm_debug_info()
Out[5]:
{'JAVA_HOME environment variable': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home',
 'jvm version': (17, 0, 10),
 'default jvm path': '/opt/homebrew/Cellar/openjdk@17/17.0.10/libexec/openjdk.jdk/Contents/Home/lib/libjli.dylib'}

In [6]: jars = py5_tools.get_classpath().split(':')

In [7]: jars
Out[7]: ['']

[hx2A]

No, the classpath doesn't get set until the JVM starts, and since it crashes for you in Jupyter, you can't get the classpath by using Jupyter. Sorry if that wasn't clear.

The first step is this, and run it in an environment where it works (IPython or generic Python interpreter)

import py5_tools
import py5

jars = py5_tools.get_classpath().split(':')

Once you have the list of Jars that are supposed to be on the classpath, try adding them to JPype without importing py5 in Jupyter, where you are getting the crash:

import jpype

for jar in jars:
    jpype.addClassPath(jar)

jpype.startJVM()
print(jpype.isJVMStarted())

[hx2A]

Well, I have some news, I just created a new env. Reinstalled everything from scratch with Java17 activated (instead of 21) at the moment of the installation.

Also, can you try creating a new environment and not using homebrew for Java? You can install Java with install-jdk, as outlined on py5's install py5 page.

And finally, it might be worth trying the install with Anaconda. Anaconda does more than manage Python libraries, it can also manage the environment, including dll files for Windows, so files for Linux, and dylib files, as is the case for MacOS.

[ivanlen]

No, the classpath doesn't get set until the JVM starts, and since it crashes for you in Jupyter, you can't get the classpath by using Jupyter. Sorry if that wasn't clear.

The first step is this, and run it in an environment where it works (IPython or generic Python interpreter)

import py5_tools
import py5

jars = py5_tools.get_classpath().split(':')

Once you have the list of Jars that are supposed to be on the classpath, try adding them to JPype without importing py5 in Jupyter, where you are getting the crash:

import jpype

for jar in jars:
    jpype.addClassPath(jar)

jpype.startJVM()
print(jpype.isJVMStarted())

Ok. I just tried this, and it is working:

jars = [
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/py5.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/jogl-all.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/gluegen-rt.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/core.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/svg/batik.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/svg/svg.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/pdf/itext.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/pdf/pdf.jar',
'/Users/***/miniconda3/envs/py5coding/lib/python3.11/site-packages/py5/jars/dxf/dxf.jar',
]

import jpype

for jar in jars:
    jpype.addClassPath(jar)

jpype.startJVM()
print(jpype.isJVMStarted())

jpype.isJVMStarted() outputs True, no crashes.

[ivanlen]

Well, I have some news, I just created a new env. Reinstalled everything from scratch with Java17 activated (instead of 21) at the moment of the installation.

Also, can you try creating a new environment and not using homebrew for Java? You can install Java with install-jdk, as outlined on py5's install py5 page.

And finally, it might be worth trying the install with Anaconda. Anaconda does more than manage Python libraries, it can also manage the environment, including dll files for Windows, so files for Linux, and dylib files, as is the case for MacOS.

Now I am with this one:

• I was already using conda to manage the environment.
• I just created a new fresh env, installed everything using conda and your yaml file (conda env create -n py5coding -f http://py5coding.org/files/install/py5_environment.yml)
• installed java using python -c "import jdk; print('Java installed to', jdk.install('17'))"
• Here I needed to set JAVA_HOME var to the new installation /Users/***/.jdk/jdk-17.0.11+9/Contents/Home (this is where pip jdk installed)

In [3]: py5_tools.get_jvm_debug_info()
Out[3]:
{'JAVA_HOME environment variable': '/Users/***/.jdk/jdk-17.0.11+9/Contents/Home',
 'jvm version': (17, 0, 11),
 'default jvm path': '/Users/***/.jdk/jdk-17.0.11+9/Contents/Home/lib/libjli.dylib'}

is working.

Now lets see the notebook 🤞 ...

NOPE:

2024-05-14 09:16:55.928 python[38222:1670408] *** Assertion failure in +[NSEvent otherEventWithType:location:modifierFlags:timestamp:windowNumber:context:subtype:data1:data2:], NSEvent.m:653
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: _NSEventMask64FromType(type) & WeirdMask'

🥀

Happy to help with any other info.

[ivanlen]

Also, doing some research I found

• [Bug]: M1 Mac, maplotlib interactive fails jupyterlab/jupyterlab#12164
• [Bug]: M1 Mac, interactive fails in jupyter notebook and jupyter lab matplotlib/matplotlib#22545
• %matplotlib osx crashses ipython/ipykernel#1124
  which might be related.
  With my current venv %matplotlib osx also makes the kernel crash.

So it might be something not releated to py5 but more general. This is bad news because you were super responsive and in these other projects devs don't see to be that responsive.

[hx2A]

Also, doing some research I found

* [[Bug]: M1 Mac, maplotlib interactive fails jupyterlab/jupyterlab#12164](https://github.com/jupyterlab/jupyterlab/issues/12164)

* [[Bug]: M1 Mac, interactive fails in jupyter notebook and jupyter lab matplotlib/matplotlib#22545](https://github.com/matplotlib/matplotlib/issues/22545)

* [%matplotlib osx crashses ipython/ipykernel#1124](https://github.com/ipython/ipykernel/issues/1124)
  which might be related.
  With my current venv `%matplotlib osx` also makes the kernel crash.

Interesting, thank you for doing this research. @villares , this is an important clue for investigating future MacOS problems: if py5 crashes Jupyter, we should ask them to test the %matplotlib osx magic.

The root problem is that on MacOS the operating system requires the GUI to run on the process's main thread. Jupyter also needs the main thread for it to run itself. The %gui osx thing enables Jupyter to share the main thread with the GUI.

When py5 runs in Jupyter it depends on %gui osx to function. Outside of Jupyter, it uses the pyobjc library, which is basically a Python wrapper of Objective C stuff. Perhaps pyobjc does not have the same problem or shortcoming that %gui osx and %matplotlib osx have.

So it might be something not releated to py5 but more general. This is bad news because you were super responsive and in these other projects devs don't see to be that responsive.

Right, this is not a py5 problem, but I did think of an idea. This will take more effort, but it will work. Consider installing Docker and running Jupyter Notebook through that. There already are Docker containers for Jupyter; this is how mybinder works. You'll need to install some py5 specific things in the Docker container to get it to work. You can pick out the relevant information in the binder directory of the py5examples repo (https://github.com/py5coding/py5examples/tree/main/binder). This will install a "virtual frame buffer" which is basically a fake display. This will work but it will run everything on the CPU and will not take advantage of the integrated graphics or GPU capabilities of your M3 machine. It might be possible to give Docker access to those resources outside of the container. I know you can do this on Linux but haven't done it myself. If you like this approach and you get the virtual frame buffer working and you think it is slow, you might want to investigate those options. In the mean time, give Docker a try and see if it meets your needs.

[ivanlen]

Apparently this is a iPython kernel problem. There is a proposed solution to address the problem. Let see what happens when this is merged.

[hx2A]

Excellent, thank you @ivanlen. I will keep an eye on this PR. When it gets released, it will make things easier for py5 users with the Apple Silicon computers.