Keep Running a tweepy stream for a long time

841 Views Asked by At

I'm running a python script that is supposed to listen tweets with a specific '@' (and then execute some code). My code is almost running 100% correctly but I still have an issue: my program stops running at random time for "no reason". Here is the error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 551, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/usr/local/lib/python3.8/http/client.py", line 518, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/http/client.py", line 583, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/usr/local/lib/python3.8/http/client.py", line 553, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 436, in _error_catcher
    yield
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 518, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.8/http/client.py", line 454, in read
    n = self.readinto(b)
  File "/usr/local/lib/python3.8/http/client.py", line 488, in readinto
    return self._readinto_chunked(b)
  File "/usr/local/lib/python3.8/http/client.py", line 599, in _readinto_chunked
    raise IncompleteRead(bytes(b[0:total_bytes]))
http.client.IncompleteRead: IncompleteRead(292 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 133, in <module>
    myStream.filter(track=['@DownDropship'])
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 474, in filter
    self._start(is_async)
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 389, in _start
    self._run()
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 320, in _run
    six.reraise(*exc_info)
  File "/usr/local/lib/python3.8/site-packages/six.py", line 703, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 289, in _run
    self._read_loop(resp)
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 339, in _read_loop
    line = buf.read_line()
  File "/usr/local/lib/python3.8/site-packages/tweepy/streaming.py", line 200, in read_line
    self._buffer += self._stream.read(self._chunk_size)
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 540, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.8/site-packages/urllib3/response.py", line 454, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(292 bytes read)', IncompleteRead(292 bytes read))

Here is a piece of my code:

    from tweepy import StreamListener
    from retrying import retry
    import tweepy

    @retry
    class MyStreamListener(StreamListener):


        @retry
        def on_status(self, status):
            do something ...
            except Exception as e:
                pass

    myStreamListener = MyStreamListener()
    myStream = tweepy.Stream(auth=twitter_handler.api.auth, listener=myStreamListener)
    myStream.filter(track=['@Something'])

I found these "@retry" here but apparently, it doesn't work properly.

Thanks for your help

0

There are 0 best solutions below