Tweepy not returning the given amount of tweets

Question

I have the following code

api = tweepy.API(auth,wait_on_rate_limit=True)

for tweet in tweepy.Cursor(api.search,
                            tweet_mode="extended",
                            q=query + " exclude:retweets").items(11000):
    hashtags = "#" + " #".join([hashtag['text'] for hashtag in tweet.entities.get('hashtags')])
    print(i)

    if tweet.place:
        tweet_place = tweet.place.full_name + ', ' + tweet.place.country_code
    else:
        tweet_place = "Not Geo-tagged"
    i += 1
    
    csvWriter.writerow([tweet.id, tweet.full_text.encode('utf-8'), tweet.created_at, tweet.lang, tweet.retweet_count, tweet.favorite_count, tweet_place, tweet.user.id, tweet.user.screen_name, tweet.user.followers_count, tweet.user.friends_count, tweet.user.created_at, tweet.user.favourites_count, tweet.user.statuses_count, tweet.user.lang, tweet.user.verified, tweet.user.location])

I am trying to get 11000 tweets with a specific search query but after some time it throws the following error:

Traceback (most recent call last):
  .............
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  .............
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  .............
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  .............
tweepy.error.TweepError: Failed to send request: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

Earlier it was happening around 2500 tweets but on changing the query it started happening at around 5000 tweets. Any idea what can be wrong and how I can fix it?

Answer 1

its most likely because you have exceeded the allowed amount of tweets that you can pull at once / per 15 minutes

check here for more information.