Django grequests batch call fail

23 Views Asked by At

I have 100 API to pin in Django, I tried to use grequests to do batch call, the batch call code looks like this:

class BatchCall:
    def __init__(self, urls, BO_urls):
        self.urls = urls
        self.BO_urls = BO_urls

    def exception(self, request, exception):
        print("Problem: {}: {}".format(request.url, exception))

    def asyncCall_release(self):
        return grequests.map(
            (grequests.get(u, auth=HTTPBasicAuth(username, password), verify=False) for u in self.urls),
            exception_handler=self.exception, size=10)

    def asyncCall_BO(self):
        return grequests.map(
            (grequests.get(u, auth=HTTPBasicAuth(username, password), verify=False) for u in self.BO_urls),
            exception_handler=self.exception, size=10)

test = BatchCall(urls, BO_urls)
# here we collect the results returned by the async function
results_release = test.asyncCall_release()
results_BO = test.asyncCall_BO()

When I test all the stuff in Jupyternotebook, everything goes well, it can give me the result, but when I run it in Djgano, I use this to run the app

python manage.py runserver

The whole app will exit itself in Django after pin 1 batch (10 api) with no reason.
*I use Pytharm to run Django

InsecureRequestWarning: Unverified HTTPS request is being made to host xxxxx. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn(

It will have 10 warning looks like this but it is normal, when I do not use batch call, it will have this warning as well.

I really have no idea what's the reason for this and it's hard to find the people who have same issues with me in Stack Overflow. Any help is really appreciated!

0

There are 0 best solutions below