How to send multiple HTTP requests using Python

Question

I'm trying to create a script that checks all possible top level domain combinations based on a given root domain name.

What I've done is generate a list of all possible TLD combinations and what I want to do is make multiple HTTP requests, analyze its results and determine whether a root domain name has multiple active top level domains or not.

Example: A client of mine has this active domains:

• domain.com
• domain.com.ar
• domain.ar

I've tried using grequests but get this error:

TypeError: 'AsyncRequest' object is not iterable

Code:

import grequests


responses = grequests.get([url for url in urls])
grequests.map(responses)

Answer 1

As I mentioned, you cannot put code as a parameter. What you want is to add a list and you can using an inline for loop, like this: [url for url in urls]

It is called list comprehensions and more information about this can be found over here: https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

So I just added two brackets:

import grequests

responses = (grequests.get(u) for u in urls)
grequests.map(responses)