Scrapy crawlera authentication issue

Question

I've been trying to use scrapy-crawlera as a proxy for scraping some data with scrapy. I've added these rows in settings.py:

DOWNLOADER_MIDDLEWARES = { 'scrapy_crawlera.CrawleraMiddleware': 610, } 
CRAWLERA_ENABLED = True 
CRAWLERA_APIKEY = 'MY_CRAWLERA_API_KEY'

In the newly changed scrapinghub website app.zyte.com i have created a Smart proxy manager account and my api key is from my account settings.

I get this warning WARNING: Retrying crawlera request for authentication issue and the websites doesn't get scraped.

Is there something I'm doing wrong?

Answer 1

This may seem like an obvious answer but I had this issue and solved it by making sure the API key I used was correct. In my case, the first API key Zyte issued to me wasn't activated. I went back to my account console and copied the API key there (it was different).

So, ensure you're using the right key or try creating a new one in the console.