skip downloading but not other tasks in scrapy pipeline

74 Views Asked by At

Is there a way I can skip downloading a webpage but still have other parts of pipeline after it execute?

Currently, I read a file of json objects in start_requests, each json object has a website URL and other data fields. if a website URL is not empty, it yields a request object else skips it.

In another function parse, I create item object after which the database pipeline comes into picture.

I want to insert the other data fields even if website url is empty in which case start_requests doesn't create a request object.

0

There are 0 best solutions below