Don't let the outside chain join the team, only save

22 Views Asked by At

I want to crawl all the links on the site including the external link, but I don't want to add the external link to the request queue, just save the external link. Dataset.pushData Using enqueueLinks will automatically add all urls to the queue, how to make the external chain not queue, only save to Dataset.pushData, that is, only the url under the main domain name of the website queue request.

I want to crawl all the urls of the website including the external chain.

0

There are 0 best solutions below