Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

redbot_cli crashes #305

Open
JannisBush opened this issue Jan 6, 2023 · 12 comments
Open

redbot_cli crashes #305

JannisBush opened this issue Jan 6, 2023 · 12 comments
Labels
bug

Comments

@JannisBush
Copy link
Contributor

I used redbot for a while on mac with python 3.9 and had no issues.
Today, I tried it out an an Ubuntu 18 machine (python 3.9) and it only works in roughly every third try.
The other runs crash with two different errors:

  1. Error:
Traceback (most recent call last):
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 153, in handle_writable
    sent = self.socket.send(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/spec_test/redbot/bin/redbot_cli", line 74, in <module>
    main()
  File "/data/data/spec_test/redbot/bin/redbot_cli", line 66, in main
    thor.run()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 116, in run
    self._run_fd_events()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 357, in _run_fd_events
    self._fd_event(event, fileno)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 190, in _fd_event
    self._fd_targets[fd].emit(event)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/events.py", line 89, in emit
    ev(*args)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 158, in handle_writable
    self._handle_close()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 202, in _handle_close
    self._close()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 208, in _close
    self.unregister_fd()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 57, in unregister_fd
    self._loop.unregister_fd(self._fd)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 339, in unregister_fd
    self._epoll.unregister(fd)
OSError: [Errno 9] Bad file descriptor
  1. Error:
exception calling callback for <Future at 0x7ff97bc8d340 state=finished returned list>
Traceback (most recent call last):
  File "/home/jannis/.pyenv/versions/3.9.6/lib/python3.9/concurrent/futures/_base.py", line 329, in _invoke_callbacks
    callback(self)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/dns/__init__.py", line 25, in done
    cb(ff.result())
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/http/client.py", line 201, in _handle_dns
    self._initiate_connection()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/http/client.py", line 220, in _initiate_connection
    tcp_client.connect_dns(
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 349, in connect_dns
    self.handle_socket_error(socket.error(err, os.strerror(err)))
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 376, in handle_socket_error
    self.handle_conn_error(err_type, err_id, err_str)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/tcp.py", line 389, in handle_conn_error
    self.unregister_fd()
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 57, in unregister_fd
    self._loop.unregister_fd(self._fd)
  File "/home/jannis/.cache/pypoetry/virtualenvs/spec-test-meXfqZJN-py3.9/lib/python3.9/site-packages/thor/loop.py", line 339, in unregister_fd
    self._epoll.unregister(fd)
FileNotFoundError: [Errno 2] No such file or directory
@mnot
Copy link
Owner

mnot commented Jan 16, 2023

What kernel are you running?

@JannisBush
Copy link
Contributor Author

I was on Linux 4.15.0-202-generic, but I updated to Linux 5.4.0-136-generic today and the issue persists.

@JannisBush
Copy link
Contributor Author

I installed everything in a fresh operation system and get another similar issue:

>>> a = subprocess.check_output(["python", "bin/redbot_cli", "-a", "-o", "har", "http://example.org/"])
Traceback (most recent call last):
  File "/home/exp/spec_test/redbot/bin/redbot_cli", line 74, in <module>
    main()
  File "/home/exp/spec_test/redbot/bin/redbot_cli", line 66, in main
    thor.run()
  File "/home/exp/.cache/pypoetry/virtualenvs/spec-test-QULprG1B-py3.10/lib/python3.10/site-packages/thor/loop.py", line 116, in run
    self._run_fd_events()
  File "/home/exp/.cache/pypoetry/virtualenvs/spec-test-QULprG1B-py3.10/lib/python3.10/site-packages/thor/loop.py", line 357, in _run_fd_events
    self._fd_event(event, fileno)
  File "/home/exp/.cache/pypoetry/virtualenvs/spec-test-QULprG1B-py3.10/lib/python3.10/site-packages/thor/loop.py", line 190, in _fd_event
    self._fd_targets[fd].emit(event)
  File "/home/exp/.cache/pypoetry/virtualenvs/spec-test-QULprG1B-py3.10/lib/python3.10/site-packages/thor/events.py", line 89, in emit
    ev(*args)
  File "/home/exp/.cache/pypoetry/virtualenvs/spec-test-QULprG1B-py3.10/lib/python3.10/site-packages/thor/tcp.py", line 135, in handle_readable
    data = self.socket.recv(self.read_bufsize)
OSError: [Errno 101] Network is unreachable
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', 'bin/redbot_cli', '-a', '-o', 'har', 'http://example.org/']' returned non-zero exit status 1.

@mnot
Copy link
Owner

mnot commented Jan 29, 2023

Using a download of the latest source, try this:

make cli https://www.mnot.net/

@JannisBush
Copy link
Contributor Author

I just realized the errors only occur when using http:// (then it fails almost always and only succeeds roughly every fifth run). When using https:// there is no problem and every call works.
The same is true when using make cli instead of the old command.

@mnot
Copy link
Owner

mnot commented Jan 31, 2023 •
edited

Hm. I'm not seeing that, using:

make cli http://www.mnot.net/

... on a 6.1.8 kernel (arch) using Python 3.10.9.

You're using 5.4.0 and 3.9.something.

I did a quick install of Ubuntu 18.04 LTS with kernel 4.15.0-202-generic and Python 3.9.16, and didn't see any issues.

Same on 20.04 LTS, with a 5.4.0-137-generic kernel and Python 3.9.16.

Is there something unusual about your kernel, network setup, etc.?

@JannisBush
Copy link
Contributor Author

The new install is using Ubuntu 22.04 LTS with a 4.19 kernel and Python 3.10.6.

The first system is inside a VM, the second one inside an lxc container. Both run on the same server.
Not sure, if there is anything unusual about the network setup.

@cageyv
Copy link
Contributor

cageyv commented Sep 13, 2023

@JannisBush do you have docker or any other container runtime?
Probably dockerized version of redbot-cli could help you

@mnot mnot added the bug label Sep 19, 2023
@mnot
Copy link
Owner

mnot commented Nov 5, 2023

Just checking in - are you still seeing this?

@JannisBush
Copy link
Contributor Author

I just checked on both machines (now updated to 6.1.38) and both still crash on HTTP with the respective error messages.

  • OSError: [Errno 101] Network is unreachable with python3.10 and inside lxd container
  • OSError: [Errno 9] Bad file descriptor with python3.9 and inside a VM

@mnot
Copy link
Owner

mnot commented Nov 6, 2023

Can you try:

pip install thor # or eqivalent
python -m thor.http.client https://www.example.com/

?

@JannisBush
Copy link
Contributor Author

JannisBush commented Nov 6, 2023 •
edited

Sure:

$ python -m thor.http.client http://www.example.com/
<frozen runpy>:128: RuntimeWarning: 'thor.http.client' found in sys.modules after import of package 'thor.http', but prior to execution of 'thor.http.client'; this may result in unpredictable behaviour
*** ERROR: DNS Error (Name or service not known)

Update:

python -m thor.http.client http://www.example.org/
<frozen runpy>:128: RuntimeWarning: 'thor.http.client' found in sys.modules after import of package 'thor.http', but prior to execution of 'thor.http.client'; this may result in unpredictable behaviour
exception calling callback for <Future at 0x7f32d78d3da0 state=finished returned list>
Traceback (most recent call last):
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/concurrent/futures/_base.py", line 340, in _invoke_callbacks
    callback(self)
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/dns/__init__.py", line 25, in done
    cb(ff.result())
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/http/client.py", line 202, in _handle_dns
    self._initiate_connection()
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/http/client.py", line 221, in _initiate_connection
    tcp_client.connect_dns(
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/tcp.py", line 349, in connect_dns
    self.handle_socket_error(socket.error(err, os.strerror(err)))
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/tcp.py", line 376, in handle_socket_error
    self.handle_conn_error(err_type, err_id, err_str)
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/tcp.py", line 389, in handle_conn_error
    self.unregister_fd()
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/loop.py", line 57, in unregister_fd
    self._loop.unregister_fd(self._fd)
  File "/home/jannis/.pyenv/versions/3.12.0/lib/python3.12/site-packages/thor/loop.py", line 339, in unregister_fd
    self._epoll.unregister(fd)
FileNotFoundError: [Errno 2] No such file or directory

Other container with python3.11:

python -m thor.http.client http://www.example.org/
<frozen runpy>:128: RuntimeWarning: 'thor.http.client' found in sys.modules after import of package 'thor.http', but prior to execution of 'thor.http.client'; this may result in unpredictable behaviour
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/http/client.py", line 660, in <module>
    test_client(sys.argv[1].encode("ascii"), sys.stdout.buffer.write, sys.stderr.write)
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/http/client.py", line 654, in test_client
    run()
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/loop.py", line 116, in run
    self._run_fd_events()
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/loop.py", line 357, in _run_fd_events
    self._fd_event(event, fileno)
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/loop.py", line 190, in _fd_event
    self._fd_targets[fd].emit(event)
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/events.py", line 89, in emit
    ev(*args)
  File "/home/ubuntu/miniconda3/lib/python3.11/site-packages/thor/tcp.py", line 153, in handle_writable
    sent = self.socket.send(data)
           ^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 101] Network is unreachable

I also updated to python3.12 and the original issue persists:

$ make cli http://www.mnot.net/
PYTHONPATH=.:./.venv/bin ./.venv/bin/python redbot/cli.py http://www.mnot.net/
Traceback (most recent call last):
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/tcp.py", line 153, in handle_writable
    sent = self.socket.send(data)
           ^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/data/spec_test/redbot_new/redbot/cli.py", line 72, in <module>
    main()
  File "/data/data/spec_test/redbot_new/redbot/cli.py", line 64, in main
    thor.run()
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/loop.py", line 116, in run
    self._run_fd_events()
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/loop.py", line 357, in _run_fd_events
    self._fd_event(event, fileno)
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/loop.py", line 190, in _fd_event
    self._fd_targets[fd].emit(event)
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/events.py", line 89, in emit
    ev(*args)
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/tcp.py", line 158, in handle_writable
    self._handle_close()
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/tcp.py", line 202, in _handle_close
    self._close()
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/tcp.py", line 208, in _close
    self.unregister_fd()
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/loop.py", line 57, in unregister_fd
    self._loop.unregister_fd(self._fd)
  File "/data/data/spec_test/redbot_new/.venv/lib/python3.12/site-packages/thor/loop.py", line 339, in unregister_fd
    self._epoll.unregister(fd)
OSError: [Errno 9] Bad file descriptor
Makefile:56: recipe for target 'cli' failed
make: *** [cli] Error 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mnot @JannisBush @cageyv