How i can block "grequests" when i found "target file"? I need to analyze like 1000 links, but just one is that good.
If I have found that link after e.g. 10 URLs, I need to block the grequests but for now him still continue until it analyze all the links.
How I can edit or fix it? Thanks
#!/usr/bin/python
import grequests
urls = [
'http://example.com/1.php',
'http://example.com/2.php',
'http://example.com/3.php',
'http://example.com/4.php'
]
def do_something(response, *args, **kwargs):
print response
print response.url
if "html" in response.text:
print response.url
async_list = []
for u in urls:
action_item = grequests.get(u, timeout=10, hooks = {'response' : do_something})
async_list.append(action_item)
grequests.map(async_list,size=10)