Getting "Connection broken: ConnectionResetError(104, 'Connection reset by peer')" error while streaming

[Forium777]

Hello, I've got this error sometime while trying to stream, it happens randomly during the stream but I can't figure out where it's from and how to handle it. The first line of the error is the last line in my code. How can I restart the stream ? Returning True on def on_exception(...): would do the trick ?

My code :

class TweetStreamListener(StreamListener):
    def on_status(self, status):
        ...

    def on_error(self, status_code):
        if status_code == 420:
            return False  # returning False in on_error disconnects the stream
        return True

    def on_exception(self, exception):
        print("[x] On_exception: " + str(exception))

Error :

[x] On_exception: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 360, in _error_catcher
    yield
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 442, in read
    data = self._fp.read(amt)
  File "/usr/lib/python3.7/http/client.py", line 447, in read
    n = self.readinto(b)
  File "/usr/lib/python3.7/http/client.py", line 481, in readinto
    return self._readinto_chunked(b)
  File "/usr/lib/python3.7/http/client.py", line 576, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/usr/lib/python3.7/http/client.py", line 544, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/usr/lib/python3.7/http/client.py", line 504, in _read_next_chunk_size
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.7/ssl.py", line 1052, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.7/ssl.py", line 911, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tweepy/streaming.py", line 300, in _run
    six.reraise(*exc_info)
  File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/local/lib/python3.7/dist-packages/tweepy/streaming.py", line 269, in _run
    self._read_loop(resp)
  File "/usr/local/lib/python3.7/dist-packages/tweepy/streaming.py", line 319, in _read_loop
    line = buf.read_line()
  File "/usr/local/lib/python3.7/dist-packages/tweepy/streaming.py", line 181, in read_line
    self._buffer += self._stream.read(self._chunk_size)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 459, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 378, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

Thanks !

[officiallymarky]

You are likely being throttled.
Push your work to a new thread so you are doing as little as possible as tweets come in and process the tweets on another thread or from a DB.

[eduardoltorres]

I am having the same issue.

I believe that I have implemented @officiallymarky's suggestion by setting the is_async parameter on the <stream>.filter() method to True; according to Tweepy's docs this parameter does exactly that, running the stream on a new thread. Still the same issue.

[Harmon758]

This should now be resolved with 68e19cc.

@eduardoltorres Running the stream on a new thread means the entire stream, including calls to your StreamListener methods, runs on that new thread, not that you're processing Tweets on a separate thread from the stream itself.
If you're looking to stream asynchronously, see #732 and #1491.

[ethankershner]

I am still having this issue. I attempted to resolve the issue by having the only work done in the on_status function be to add the status to a list, which is then processed later periodically. This does not seem to resolve the issue, however. I am running the stream with the async parameter set to 'True'.

[Harmon758]

What version of Tweepy are you using?

[ethankershner]

What version of Tweepy are you using?

3.10.0.

[Harmon758]

The commit that fixed this, 68e19cc, is on the master branch, but hasn't been released yet.
You'll have to either install from the master branch or wait for v4.0.