send multiple requests simultaneously using CONCURRENCY_REQUEST using Scrapy

282 Views Asked by At

Please, How to configure exactly the CONCURRENCY_REQUEST, CONCURRENCY_REQUEST_PER_DOMAIN for to send multiple requests simultaneously ? My DOWNLOAD_DELAY = 5 and CONCURRENCY_REQUEST_PER_IP = 1

When I set CONCURRENCY_REQUEST = 4 there is no difference in time and I don't understand

1

There are 1 best solutions below

1
On

You mistyped name of settings
there is no CONCURRENCY_REQUEST , CONCURRENCY_REQUEST_PER_DOMAIN and CONCURRENCY_REQUEST_PER_IP settings in scrapy.

You need to use CONCURRENT_REQUESTS , CONCURRENT_REQUESTS_PER_DOMAIN
CONCURRENT_ not CONCURRENCY_