Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows Python wheels: Bad address src\epoll.cpp:100 #1505

Open
zhanggyarcher opened this issue Feb 11, 2021 · 45 comments · Fixed by #1758
Open

Windows Python wheels: Bad address src\epoll.cpp:100 #1505

zhanggyarcher opened this issue Feb 11, 2021 · 45 comments · Fixed by #1758

Comments

@zhanggyarcher
Copy link

Hello, recently I set up a new python environment on a new computer as usual, I got automatically installed pyzmq==21.0.2, and I cannot use jupyter notebook, Jupyter lab, vs code at all, the python kernel crashes every time when I open a editor, and after one second, it directly crashes and lose connection to the kernel.

This problem annoyed me for several weeks, and today I saw many people encountered this issue From December 2020 to January 2021. And I have never encountered this issue in the past.

Also, this error message is somewhat not useful and not making sense, finally I realized its pyzmq error.
I degraded it to pyzmq==19.0.2 , it finally resolved this issue.

I hope this could be fixed, or if you could push old versions of pyzmq for users to install as dependencies.
Thank you very much!
IMG_1690

@minrk
Copy link
Member

minrk commented Feb 11, 2021

Can you test with current pyzmq 22.0.2? As far as I am aware, all of these issues are fixed in the latest release.

pyzmq has no control over error messages from the C runtime thrown by the underlying libzmq.

@zhanggyarcher
Copy link
Author

Can you test with current pyzmq 22.0.2? As far as I am aware, all of these issues are fixed in the latest release.

pyzmq has no control over error messages from the C runtime thrown by the underlying libzmq.

Sure, I will have a look and let you know. But could you stop letting them push 21.0.2 as dependencies? Thanks!

@minrk
Copy link
Member

minrk commented Feb 11, 2021

But could you stop letting them push 21.0.2 as dependencies?

Who are 'they'? If you pip install pyzmq you should get 22.0.2. That's all I have control over.

@zhanggyarcher
Copy link
Author

But could you stop letting them push 21.0.2 as dependencies?

Who are 'they'? If you pip install pyzmq you should get 22.0.2. That's all I have control over.

When I install pyzmq==22.0.2, it still has this crash problem. Then I reverted to 19.0.2, the problem resolved. Everything is working fine.
Screenshot_1
Screenshot_2

When I check dependency list, ipykernel is calling to install pyzmq, when user is installing jupyter or jupyter-lab, pyzmq==21.0.2 is automatically installed, so I hope you could let them to install pyzmq==19.0.2, not the later version.
Thank you!
Screenshot_3

@minrk
Copy link
Member

minrk commented Feb 12, 2021

Thank you for testing and confirming this is an issue in the current version!

Can you run:

python3 -m pip install pytest pyzmq==22.0.2
python3 -m pytest -v -m "not wheel and not new_console" --pyargs zmq.tests

and share the output?

Also, can you share some more information about your environment, so I can learn a bit about what makes your environment different from all of our tests?

  • Windows Version
  • Python version, and how it is installed (from Python.org, conda-forge, etc.)

@bluca any hints as to why this epoll_ctl assert would fail on Windows? This is using the libzmq-v141-x64-4_3_4 artifact on bintray.

so I hope you could let them to install pyzmq==19.0.2, not the later version.

I'm sorry, I really don't know what you mean by this. Anyone is free to install any version, so this is already the case.

@minrk minrk changed the title Installation of pyzmq==21.0.2 cause Python kernel to crash Windows Python 3.6 wheel: Bad address src\epoll.cpp:100 Feb 12, 2021
@bluca
Copy link
Member

bluca commented Feb 12, 2021

  • Windows Version
  • Python version, and how it is installed (from Python.org, conda-forge, etc.)

@bluca any hints as to why this epoll_ctl assert would fail on Windows? This is using the libzmq-v141-x64-4_3_4 artifact on bintray.

Sorry, don't know - maybe @sigiesec can help

@zhanggyarcher
Copy link
Author

@minrk Sorry for the late reply, I will try my best to test these on a new computer as soon as I can(to avoid damaging my environment).
For this issue, I encontered on Windows 10 1909, 20H2, python 3.6.8, 3.7.9, 3.8.6, 3.8.7, so this is not only the issue for python 3.6. So I am not sure the modified title for this issue is suitable or not. And I tried all these versions and I reinstalled windows10 and the python environment more than 20 times, this issue always appear.
I installed python 3.6.8 or other versions via windows x86_64 installer, and then I use pip to install these packages sequentially: tensorflow, tensorflow-gpu, matplotlib, scipy,scikit-learn,torch, jupyterlab. As what I said above, the jupyter need ipykernel, and ipykernel needs pyzmq. When I install jupyterlab or jupyter-notebook, it automatically install ipykernel and automatically install pyzmq=21.0.2, I have no choice. And many users don't know they installed pyzmq, and they never know the version. I manually degraded to pyzmq=19.0.2, it solves the issue.
My friend tried to use anaconda to help me to install the environment, it is ok to open the jupyter to run "hello world", but the tensorflow will fail, it's weird and it's another story.

Thank you!

All the package versions are listed below.
`
PS C:\Windows\system32> pip list
Package Version


absl-py 0.11.0
anyio 2.1.0
argon2-cffi 20.1.0
astor 0.8.1
astunparse 1.6.3
async-generator 1.10
attrs 20.3.0
backcall 0.2.0
bleach 3.3.0
cached-property 1.5.2
cachetools 4.2.1
certifi 2020.12.5
cffi 1.14.4
chardet 4.0.0
colorama 0.4.4
contextvars 2.4
cryptography 3.2.1
cycler 0.10.0
dataclasses 0.8
decorator 4.4.2
defusedxml 0.6.0
entrypoints 0.3
flatbuffers 1.12
future 0.18.2
gast 0.3.3
google-auth 1.25.0
google-auth-oauthlib 0.4.2
google-pasta 0.2.0
grpcio 1.32.0
h5py 2.10.0
idna 2.10
immutables 0.15
importlib-metadata 3.4.0
ipykernel 5.4.3
ipython 7.16.1
ipython-genutils 0.2.0
jedi 0.18.0
Jinja2 2.11.3
json5 0.9.5
jsonschema 3.2.0
jupyter-client 6.1.11
jupyter-core 4.7.1
jupyter-kite 1.2.0
jupyter-server 1.3.0
jupyterlab 2.2.9
jupyterlab-pygments 0.1.2
jupyterlab-server 1.2.0
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.2
kiwisolver 1.3.1
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.3.4
mistune 0.8.4
mysqlclient 2.0.3
nbclient 0.5.2
nbconvert 6.0.7
nbformat 5.1.2
nest-asyncio 1.5.1
nodejs 0.1.1
notebook 6.2.0
npm 0.1.1
numpy 1.19.5
oauthlib 3.1.0
opt-einsum 3.3.0
optional-django 0.1.0
packaging 20.9
pandas 1.1.5
pandocfilters 1.4.3
parso 0.8.1
pickleshare 0.7.5
Pillow 8.1.0
pip 18.1
pipdeptree 2.0.0
prometheus-client 0.9.0
prompt-toolkit 3.0.14
protobuf 3.14.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.20
Pygments 2.7.4
pyparsing 2.4.7
pyrsistent 0.17.3
python-dateutil 2.8.1
pytz 2021.1
pywin32 300
pywinpty 0.5.7
pyzmq 19.0.2
requests 2.25.1
requests-oauthlib 1.3.0
rsa 4.7
Send2Trash 1.5.0
setuptools 53.0.0
six 1.15.0
sniffio 1.2.0
tb-nightly 2.5.0a20210209
tensorboard 2.4.1
tensorboard-plugin-wit 1.8.0
tensorflow 2.4.1
tensorflow-estimator 2.4.0
tensorflow-gpu 2.4.1
termcolor 1.1.0
terminado 0.9.2
testpath 0.4.4
tf-estimator-nightly 2.5.0.dev2021020901
torch 1.7.0+cpu
torchaudio 0.7.0
torchvision 0.8.1+cpu
tornado 6.1
traitlets 4.3.3
typing-extensions 3.7.4.3
urllib3 1.24.3
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.36.2
wrapt 1.12.1
zipp 3.4.0
You are using pip version 18.1, however version 21.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
`

@minrk minrk changed the title Windows Python 3.6 wheel: Bad address src\epoll.cpp:100 Windows Python wheels: Bad address src\epoll.cpp:100 Feb 18, 2021
@minrk
Copy link
Member

minrk commented Feb 18, 2021

Thanks for the info! I'll see if I can figure out how to debug it. Since it does not occur on my Windows VMs, I don't really feel like I have any actions available at the moment.

Were you able to run pyzmq tests with the commands I gave above? That might help narrow down the issue.

When I install jupyterlab or jupyter-notebook, it automatically install ipykernel and automatically install pyzmq=21.0.2, I have no choice

You do have a choice. You can always specify pinning of any package in the install command. For example:

pip install jupyterlab 'pyzmq<20'

ipykernel does not request 'pyzmq==22.0.2', it requests 'pyzmq' which gets the latest version by default. If you want to specify a different version, you can do that.

My friend tried to use anaconda to help me to install the environment

If you are using conda, it's a good idea to use conda to install packages like tensorflow and pyzmq that have DLL dependencies instead of mixing conda and pip. Tensorflow wheels in particular are known to have serious compatibility issues. For example:

conda create -n tensorflow tensorflow-gpu jupyterlab python=3.8

to create an environment called 'tensorflow' with tensorflow-gpu and jupyterlab with Python 3.8.

@zhanggyarcher
Copy link
Author

@minrk the result is:
Screenshot_1

@minrk
Copy link
Member

minrk commented Feb 22, 2021

@zhanggyarcher could you download libzmq-v141-x64-4_3_4.zip from https://dl.bintray.com/zeromq/generic/ and run test_reqrep_tcp.exe and see if you get a similar error? If you do, could you also download the v140 version and test that?

@zhanggyarcher
Copy link
Author

@minrk nothing has displayed, no error message

@zoloc
Copy link

zoloc commented May 10, 2021

Hello, recently I set up a new python environment on a new computer as usual, I got automatically installed pyzmq==21.0.2, and I cannot use jupyter notebook, Jupyter lab, vs code at all, the python kernel crashes every time when I open a editor, and after one second, it directly crashes and lose connection to the kernel.

This problem annoyed me for several weeks, and today I saw many people encountered this issue From December 2020 to January 2021. And I have never encountered this issue in the past.

Also, this error message is somewhat not useful and not making sense, finally I realized its pyzmq error.
I degraded it to pyzmq==19.0.2 , it finally resolved this issue.

I hope this could be fixed, or if you could push old versions of pyzmq for users to install as dependencies.
Thank you very much!
IMG_1690

Thanks for the solution, I've fixed it

@anupsaha1
Copy link

I am having error while installing pyzmq 19.0.2
(base) C:\Users\Cipher_X>This can take several minutes. Press CTRL-C to abort.
'This' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>failed
'failed' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>UnsatisfiableError: The following specifications were found
'UnsatisfiableError:' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>to be incompatible with the existing python installation in your environment:
'to' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>Specifications:
'Specifications:' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X> - anaconda -> python[version='2.7.13|2.7.14|2.7.14|2.7.15|2.7.15|2.7.16|3.6.10|3.6.10|3.7.10|3.7.11|3.9.7|3.8.12|3.8.8|3.9.4|3.7.9|3.8.5|3.7.7|3.8.3|3.8.1|3.7.6|3.6.9|3.7.4|3.6.8|3.7.3|3.7.3|3.6.7|3.7.1|3.7.0|3.6.6|3.6.5|3.5.5|3.6.4|3.5.4|3.6.3|3.5.4|3.6.2|3.5.4|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=2.7,<2.8.0a0|>=3.7,<3.8.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0',build='hd3c4935_11|h6679aeb_11|hb034564_12|h8c3f1cb_23|h6538335_1|h0c2934d_0|hea74fb7_0|hea74fb7_0|hcb6e200_5|h9f7ef89_2|h8c8aaf0_1|h9f7ef89_7|h5263a28_0|h5500b2f_0|h9f7ef89_0|h81c818b_4|h9f7ef89_2|h5fd99cc_1|h6244533_0|h6244533_0|hdbf39b2_5|h6244533_0|h6244533_0|h6244533_1|h60c2a47_0|he1778fa_2|h5fd99cc_8_cpython|h60c2a47_2|hcb6e200_0|h8c8aaf0_0|h8c8aaf0_6|he216670_0|h0c2934d_2|h1357f44_23|h9e2ca53_1|hdec4e59_20|h59f5a59_20']
| was unexpected at this time.
(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>Your python: python=3.9
'Your' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>If python is on the left-most side of the chain, that's the version you've asked for.
is was unexpected at this time.
(base) C:\Users\Cipher_X>When python appears to the right, that indicates that the thing on the left is somehow
'When' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>not available for the python version you are constrained to. Note that conda will not
'not' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>change your python version to a different minor version unless you explicitly specify
Invalid parameter(s)
CHANGE { LOGON | PORT | USER }

(base) C:\Users\Cipher_X>that.
'that.' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>The following specifications were found to be incompatible with each other:
'The' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>Output in format: Requested package -> Available versions
'Output' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>
(base) C:\Users\Cipher_X>Package html5lib conflicts for:
'Package' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>anaconda -> bleach==2.1.4=py27_0 -> html5lib[version='>=0.99999999|>=0.99999999pre,!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8']
'!' is not recognized as an internal or external command,
operable program or batch file.

(base) C:\Users\Cipher_X>anaconda -> html5lib[version='0.999999999|0.999999999|0.999999999|1.0.1|1.0.1|1.0.1|1.0.1|1.0.1|1.0.1|1.0.1|1.1',build='py27h47766b3_0|py36ha09b1f3_0|py27h5a33001_0|py27_0|py37_0|py_0|pyhd3eb1b0_0|py36_0|py_0|py36h047fa9f_0|py35h047fa9f_0|py35h79d4e7f_0']
'0.999999999' is not recognized as an internal or external command,
operable program or batch file.

@anupsaha1
Copy link

@kevin-bates
please have a look

@anupsaha1
Copy link

this is the message appearing when I want to run jupyter notebook(it's obvious it won't run but for understanding the error I think this will help you )
Traceback (most recent call last):
File "C:\Users\Cipher_X\anaconda3\Scripts\jupyter-notebook-script.py", line 6, in
from notebook.notebookapp import main
File "C:\Users\Cipher_X\anaconda3\lib\site-packages\notebook\notebookapp.py", line 78, in
from .services.kernels.kernelmanager import MappingKernelManager, AsyncMappingKernelManager
File "C:\Users\Cipher_X\anaconda3\lib\site-packages\notebook\services\kernels\kernelmanager.py", line 18, in
from jupyter_client.session import Session
File "C:\Users\Cipher_X\anaconda3\lib\site-packages\jupyter_client_init_.py", line 4, in
from .connect import *
File "C:\Users\Cipher_X\anaconda3\lib\site-packages\jupyter_client\connect.py", line 21, in
import zmq
ModuleNotFoundError: No module named 'zmq'

@minrk
Copy link
Member

minrk commented Jan 26, 2022

@anupsaha1 I'm not sure if this is related to the issue, but all I can see is pasting lots of non-CMD text into a CMD prompt.

ModuleNotFoundError: No module named 'zmq'

This suggests you don't have pyzmq. You can try installing it:

conda instal pyzmq

@anupsaha1
Copy link

pyzmq==19.0.2

thanks mate it's worked
it was giving me an error for the first time installing pyzmq==19.0.2 but this time it's worked
finally, after a few days of searching, I got the solution wlah!

pjaggi1 added a commit to pjaggi1/libzmq that referenced this issue May 19, 2022
zeromq/pyzmq#1505
a few Jupyter users have encountered an issue where there program ends after this assert:
Bad file descriptor (C:\ci\zeromq_1602704446950\work\src\epoll.cpp:100)

We found this happens for users that have a windows 10 build that supports AF_Unix sockets but does not for those who have older windows 10 builds.  this change is to remove the preserved errno (which doesn't appear to be used/checked anywhere-hence the later assert/abort) and instead after cleanup try to use a tcp/ip socket.
@malsesto
Copy link

Late to the party but I encountered the same issue today with version 23.0.2
System: Win 11
Python: 3.10.5

@knidarkness
Copy link

@malsesto had this same issue today, for me the solution was to rollback the Python version to 3.8.13 and then pyzmq to 19.0.2 via the Anaconda Navigator > Environments > (current environment).

@minrk
Copy link
Member

minrk commented Sep 8, 2022

@Godsix since you've been able to reproduce this in #1754 by building libzmq from source, can you build libzmq once each from commits zeromq/libzmq@b2921d7 and zeromq/libzmq@b77543a (both commits from https://github.com/zeromq/libzmq/pull/4422/commits) and report the result(s)? The first result should give an informative error, while the second hopefully actually fixes the issue.

@minrk
Copy link
Member

minrk commented Sep 15, 2022

I really hope the new wheels for pyzmq 24.0 fix this. Since I've never been able to reproduce the problem, I can't verify myself. Anyone who has been seeing this who can confirm here would be much appreciated.

@aiden-leong
Copy link

I really hope the new wheels for pyzmq 24.0 fix this. Since I've never been able to reproduce the problem, I can't verify myself. Anyone who has been seeing this who can confirm here would be much appreciated.

I have tested pyzmq 24.0 on my friend's Windows 10 PC. Unfortunately, it still crashes with JupyterLab. The only working workaround for this is downgrading to 19.0.2.

I'll let you know if there's any further helpful debugging info.

Cheers.

@minrk
Copy link
Member

minrk commented Sep 20, 2022

@aiden-leong thanks, that's helpful, if disheartening. I'm absolutely at a loss since I've set up dozens of Windows systems of various versions to test and have never been able to reproduce the problem. I can't figure out what makes these rare failing environments special, so there isn't really any way to investigate further. I simply cannot fix this, and someone else is going to have to figure it out.

Are you using 64b or 32b Windows wheels? What version of Python and how did you install Python?

Can you tell which line causes the crash in:

import zmq
ctx = zmq.Context()
pull = ctx.socket(zmq.PULL)
url = "tcp://127.0.0.1:5555"
pull.bind(url)
push = ctx.socket(zmq.PUSH)
push.connect(url)
push.send(b'test')
pull.recv()
push.close()
pull.close()
ctx.term()

Do you see similar crashes if you install pyzmq 24 through conda?

@minrk
Copy link
Member

minrk commented Sep 20, 2022

@aiden-leong when it finishes, can you try installing from the appropriate wheels produced in https://github.com/zeromq/pyzmq/actions/runs/3090754782

Maybe vc140 wheels will work where vc141 did not

@aiden-leong
Copy link

aiden-leong commented Sep 20, 2022

@aiden-leong thanks, that's helpful, if disheartening. I'm absolutely at a loss since I've set up dozens of Windows systems of various versions to test and have never been able to reproduce the problem. I can't figure out what makes these rare failing environments special, so there isn't really any way to investigate further. I simply cannot fix this, and someone else is going to have to figure it out.

Are you using 64b or 32b Windows wheels? What version of Python and how did you install Python?

Can you tell which line causes the crash in:

import zmq
ctx = zmq.Context()
pull = ctx.socket(zmq.PULL)
url = "tcp://127.0.0.1:5555"
pull.bind(url)
push = ctx.socket(zmq.PUSH)
push.connect(url)
push.send(b'test')
pull.recv()
push.close()
pull.close()
ctx.term()

Do you see similar crashes if you install pyzmq 24 through conda?

  • Windows 10 64-bit, unknown minor version.
  • Python 3.9 by conda create --name myenv python=3.9
  • Can't tell which line. To reproduce the issue: pip install jupyterlab then run jupyter-lab in terminal, run print("Hello") inside jupyterlab web page, the crash message will show in the terminal.

After upgrading to Windows 11, the issue is resolved. It should be related to some specific Windows 10 minor versions or patches.
I can not reproduce the bug anymore, but people who still have the same issue might export there Windows 10 version number and/or the patch list to find a clue.

Workaround:

  1. Downgrade to 19.0.2
  2. Upgrade to Windows 11

Cheers.

@stfmnsl
Copy link

stfmnsl commented Oct 31, 2022

@minrk I am sorry to report that I have the same issue. Brand new computer, up to date Windows 11, Python 3.10.7, latest version of jupyter and all dependencies. I can't start a kernel in jupyter jupyter-lab, it crashes immediately with Bad address (C:\projects\libzmq\src\epoll.cpp:100) which does not exist.
I am happy to engage offline to help debug the issue on my machine.

Processor 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz
Installed RAM 32.0 GB (31.7 GB usable)
Device ID 8C5E8BAA-6590-4208-A92D-0DA3829A6FEF
Product ID 00331-20561-79099-AA062
System type 64-bit operating system, x64-based processor
Pen and touch No pen or touch input is available for this display

Edition Windows 11 Pro
Version 21H2
Installed on ‎10/‎22/‎2022
OS build 22000.1165
Experience Windows Feature Experience Pack 1000.22000.1165.0

@minrk
Copy link
Member

minrk commented Oct 31, 2022

@stfmnsl thanks for testing, and that's utterly baffling. How did you install pyzmq and which version of pyzmq did you get? Can you test the win_amd64 wheels for your Python from here: https://github.com/zeromq/pyzmq/actions/runs/3090754782

@stfmnsl
Copy link

stfmnsl commented Oct 31, 2022

I did a default installation using pip, which downloaded the latest version pyzmq 24.0.1. Then I tried with 25.0.0b1. Same results.

I went into debug to see what was happening:
C:\Users\stephane.monsallier\fastai>python -m pdb test_zmq.py

c:\users\stephane.monsallier\fastai\test_zmq.py(1)()
-> import zmq
(Pdb) break c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py:157
Breakpoint 1 at c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py:157
(Pdb) run
Restarting C:\Users\stephane.monsallier\fastai\test_zmq.py with arguments:
c:\users\stephane.monsallier\fastai\test_zmq.py(1)()
-> import zmq
(Pdb) next
c:\users\stephane.monsallier\fastai\test_zmq.py(2)()
-> ctx = zmq.Context()
(Pdb) next
c:\users\stephane.monsallier\fastai\test_zmq.py(3)()
-> pull = ctx.socket(zmq.PULL)
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(157)init()
-> self._shadow = False
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(159)init()
-> super().init(
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(160)init()
-> ctx_or_socket,
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(161)init()
-> socket_type,
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(162)init()
-> shadow=shadow_address,
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(163)init()
-> copy_threshold=copy_threshold,
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(159)init()
-> super().init(
(Pdb) p shadow
0
(Pdb) p socket_type
<SocketType.PULL: 7>
(Pdb) p copy_threshold
None
(Pdb) p ctx_or_socket
<zmq.Context() at 0x1f5475876a0>
(Pdb) step
Bad address (C:\projects\libzmq\src\epoll.cpp:100)

Not sure that helps. I don't understand why the constructor gets to Bad Address.

I will try the wheels you pointed out.

@stfmnsl
Copy link

stfmnsl commented Oct 31, 2022

Same results with the wheel (24.0.0 cp130) but the debug sequence is different:

C:\Users\stephane.monsallier\fastai>python -m pdb test_zmq.py

c:\users\stephane.monsallier\fastai\test_zmq.py(1)()
-> import zmq
(Pdb) next
c:\users\stephane.monsallier\fastai\test_zmq.py(2)()
-> ctx = zmq.Context()
(Pdb) next
c:\users\stephane.monsallier\fastai\test_zmq.py(3)()
-> pull = ctx.socket(zmq.PULL)
(Pdb) step
--Call--
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(277)socket()
-> def socket(self: T, socket_type: int, **kwargs: Any) -> ST:
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(289)socket()
-> if self.closed:
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(291)socket()
-> s: ST = self._socket_class( # set PYTHONTRACEMALLOC=2 to get the calling frame
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(292)socket()
-> self, socket_type, **kwargs
(Pdb) next
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(291)socket()
-> s: ST = self._socket_class( # set PYTHONTRACEMALLOC=2 to get the calling frame
(Pdb) step
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(292)socket()
-> self, socket_type, **kwargs
(Pdb) step
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\context.py(291)socket()
-> s: ST = self._socket_class( # set PYTHONTRACEMALLOC=2 to get the calling frame
(Pdb) step
--Call--
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(93)init()
-> def init(self: "Socket[bytes]", *a, **kw):
(Pdb) step
c:\users\stephane.monsallier\appdata\roaming\python\python310\site-packages\zmq\sugar\socket.py(94)init()
-> super().init(*a, **kw)
(Pdb) step
Bad address (C:\projects\libzmq\src\epoll.cpp:100)

@stfmnsl
Copy link

stfmnsl commented Nov 4, 2022

I downloaded the latest version of libzmq (8d4f4efe) and compiled it. I manually replaced libzmq into pyzmq.lib.
I don't see the error message anymore (Bad address (C:\projects\libzmq\src\epoll.cpp:100)).
I can run successfully the small test:

import zmq
ctx = zmq.Context()
pull = ctx.socket(zmq.PULL)
url = "tcp://127.0.0.1:5555"
pull.bind(url)
push = ctx.socket(zmq.PUSH)
push.connect(url)
push.send(b'test')
pull.recv()
push.close()
pull.close()
ctx.term()

But I have different errors when trying to run jupyter:
OSError: [WinError 10038] An operation was attempted on something that is not a socket

@KianSOH
Copy link

KianSOH commented Dec 14, 2022

image

I wish to add here that for people who have issues logging in with Kernel of Jupyter Lab/Notebook, based on the simple advise of installing / downgrading, using the ANACONDA commands [For people with ANACONDA installed]

conda install pytest pyzmq==19.0.2

This will fix the problem of non linking up with the Kernel! I basically looked through all advise online, and this one works for me!

@KianSOH
Copy link

KianSOH commented Jan 9, 2023

I wish to add another findings to this frustrating event of Jupyter Notebook (Python), not able to run in browsers (Chrome, EDGE, FireFox, etc) and in Visual Studio Code. [The stupid error codes, and not able to run Jupyter Notebook locally at Visual Studio Code]

I found out that the real culprit is due to the VPN settings that most people will use.

Next time when you find yourself facing this frustrating event perform the following steps.

  1. go to windows command prompt, and run as administrator
  2. type the following command "netsh winsock reset"
  3. reboot your computer.

This should be a permanent solution, but when the VPN starts to mess up the network settings, all you need to do is to perform the above command. [pyzmq is not the true culprit, I found out that I didn't actually install version 19.0.2 since my version of Python is 3.9 and above]

@TengPan2011
Copy link

TengPan2011 commented May 27, 2023

The VPN issue sounds about right, I had a PC, freshly installed spyder on it. It works, but right after a VPN was installed, everything started failing, including fresh installations of vscode, new conda envs, non conda venvs, etc, it all traces back down to pyzmq and the epoll bad address issue.
@KianSOH 's solution worked perfectly.
1.create a new conda environment based on python 3.8,
2.conda install pytest pyzmq==19.0.2
3.Install spyder/jupyter whatever.
I could not believe VPNs would be intefering with jupyter.

@alonbl
Copy link

alonbl commented May 27, 2023

Building libzmq with the following configuration resolves the issue:

-DPOLLER=select -DZMQ_HAVE_IPC=OFF -DZMQ_CV_IMPL=stl11

@SunXinZH
Copy link

The VPN issue sounds about right, I had a PC, freshly installed spyder on it. It works, but right after a VPN was installed, everything started failing, including fresh installations of vscode, new conda envs, non conda venvs, etc, it all traces back down to pyzmq and the epoll bad address issue. @KianSOH 's solution worked perfectly. 1.create a new conda environment based on python 3.8, 2.conda install pytest pyzmq==19.0.2 3.Install spyder/jupyter whatever. I could not believe VPNs would be intefering with jupyter.

I have the same problem. After uninstalling VPN, it is ok.
I think some VPN software will forward network requests and localhost's real address is changed

@alejandrozavaletam
Copy link

I wish to add another findings to this frustrating event of Jupyter Notebook (Python), not able to run in browsers (Chrome, EDGE, FireFox, etc) and in Visual Studio Code. [The stupid error codes, and not able to run Jupyter Notebook locally at Visual Studio Code]

I found out that the real culprit is due to the VPN settings that most people will use.

Next time when you find yourself facing this frustrating event perform the following steps.

  1. go to windows command prompt, and run as administrator
  2. type the following command "netsh winsock reset"
  3. reboot your computer.

This should be a permanent solution, but when the VPN starts to mess up the network settings, all you need to do is to perform the above command. [pyzmq is not the true culprit, I found out that I didn't actually install version 19.0.2 since my version of Python is 3.9 and above]

Just another confirmation of the same issue with Jupyter and "bad address" message. After looking all around without a proper solution and reading this thread, this was the only solution. The previous approach downgrading pyzmq was not possible and gave other errors. Precisely, I'm using a VPN, w11, conda 23.7.2 and typical python 3.10 and jupyter installation

@KianSOH
Copy link

KianSOH commented Aug 11, 2023

Glad to know that my days of frustration with this silly problem have helped many online! It took me a few days of reading online and researching on the problem, until I come across some nice bloke who posted the solution online in youtube.

I tried the simple solution, and Jupyter started to work in all browsers.

  1. go to windows command prompt, and run as administrator
  2. type the following command "netsh winsock reset"
  3. reboot your computer.

I had actually reinstalled conda numerous times, and also tried with different versions of python, and nothing works... damn frustrating, then I came across this thread with the same error message, and I tried the pyzmq downgrade, it worked intermittently... the true cause of the problem is the VPN.

@aszekMosek
Copy link

Same issue here. Unfortunately any solution that involves rebooting is not an option since I am in a docker contained started by a CI pipeline. I am not able to stay on version 19.0.2 forever either. What other options are there? I don't even want to run jupyter, only to execute a notebook file from a Python script, something along the lines of:

import sys
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor as EP

with open(sys.argv[1]) as f:
    nb = nbformat.read(f, as_version=4)
    ep = EP(timeout=400, kernel_name='python3')
    try:
        ep.preprocess(nb, {'metadata' : { 'path' : '.' }})
        with open('executed_notebook.ipynb', mode='wt', encoding='utf8') as fout:
            nbformat.write(nb, fout)

If nothing else helps then I could extract the Python code from the notebook and run it with standalone python, but any more straightforward ideas will be appreciated. (I am surprised such a simple operation as this has to involve libzmq at all.)

@minrk
Copy link
Member

minrk commented Oct 20, 2023

I am surprised such a simple operation as this has to involve libzmq at all

executing any code from a notebook uses libzmq because it uses the Jupyter protocol to communicate with the kernel, which is a different process (can be a different language altogether), otherwise it wouldn't be a realistic execution of the notebook.

If you want to export the notebook to a script before running it with a plain Python interpreter, that may be an option (nbconvert --to script).

I think you might be hitting a quite different issue, though, as this is specifically about an unusual Windows-specific networking configuration issue, and I assume your CI pipeline in docker is not a Windows container? I'm not aware of any non-Windows users seeing issues like this, so more information (and a different Issue with repro steps) would be required.

@aszekMosek
Copy link

@minrk It is a windows docker (I also get it on physical windows machines). I make a miniconda environment listed below and in it run the notebook execution code mentioned in my last comment.

dependencies:
  - python==3.11
  - numpy
  - scipy
  - matplotlib
  - nb_conda_kernels
  - nbconvert
  - nbformat
  - pandas
  - statsmodels

With python==3.7 and additional pyzmq=19.0.2 it is still OK.

@minrk
Copy link
Member

minrk commented Oct 20, 2023

ah, got it. Then it is the same issue. This is a libzmq bug that pyzmq can't do anything about, but there is a chance the brand new libzmq 4.3.5 build fixes it, if you want to try adding zeromq==4.3.5 to your env to test env.

@aszekMosek
Copy link

@minrk Thanks. I will try when pyzmq is upgraded to allow it. Forcing zeromq==4.3.5 didn't work for me without breaking it all even worse.

@minrk
Copy link
Member

minrk commented Feb 16, 2024

There are now updated wheels with zeromq-4.3.5.

If anyone who can still reproduce this issue can test the wheels available here and report back, that would be a huge help.

pyzmq now has control over the cmake options of the libzmq it bundles in wheels, so there is some hope of resolving it if those wheels don't work yet.

I'm sorry I've never been able to personally reproduce this issue, so I can't verify the fix myself!

@aszekMosek
Copy link

aszekMosek commented Feb 19, 2024

I installed the new wheel in an existing environment which is specified as in one of my previous comments but with python==3.9. The installation did not affect the existing zeromq==4.3.4 I had in that environment but maybe now it doesn't matter. After that it seems to resolve the problem, ie. I can run all the notebooks I previously only could run with pyzmq==19.0.2 and the error in epool.cpp:100 does not come up any more.

The only new thing I have is this warning which is probably a different story (irrelevant for me):

\lib\site-packages\zmq\_future.py:700: RuntimeWarning: Proactor event loop does not implement add
_reader family of methods required for zmq. Registering an additional selector thread for add_reader support via tornado. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` to avoid this warni
ng.

and in one case (not deterministically reproducible) I got

Assertion failed: pfd.revents & POLLIN (C:\Users\runneradmin\AppData\Local\Temp\tmp46punxxq\build\_deps\bundled_libzmq-src\src\signaler.cp
p:238)

but it seems the execution continued regardless and completed as success.

I will keep you informed if anything should change after more thorough testing.

Many thanks for looking into this and for your help!

@minrk
Copy link
Member

minrk commented Feb 21, 2024

The proactor warning is not an issue, but could probably use some better communication (#1830)

The signaler error is concerning. Do keep us updated on if that continues to be a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.