Synchronous HTTPS requests with curlpp

673 Views Asked by At

When I try to preform lots of synchronous https request using curlpp, I get the following error:

Failed to connect to https://www.google.com/ port 443: No route to host

The code below is of course only a stripped-down, cleaned-up sample of the actual code. The url is different, but the result are the same.

void ThreadFunction()
{
   try
   {
      curlpp::Cleanup myCleanup;
      std::ostringstream os;
      os << curlpp::options::Url("https://www.google.com/");
   }
    catch( curlpp::RuntimeError &e )
    {
        std::cout << e.what() << std::endl;
    }
   catch( curlpp::LogicError &e )
    {
        std::cout << e.what() << std::endl;
   }
}

int main(void)
{
   thread threads[10];

   int loops = 0;

   while(loops < 3000)
   {
      for(int i = 0; i < 10; i++)
      {
         threads[i] = thread(ThreadFunction);
      }

      for(int i = 0; i < 10; i++)
      {
         threads[i].join();
      }
      loops++;
   }

   return 0;
}

The errors occur not direct from the beginning, but around loop ~1000. After that, each request generates this error. I have tried using different ports (like 80) for each thread but curlpp only seems to preform https request using port 443. When running only one thread everything seems fine (except it takes way too long, thus the multiple threads).

So my theory A: It is actually possible to preform synchronous request like this, but I’m missing some closing/cleaning of the port after the request. (I mean, it works for the first ~1000 * 10 requests)

Theory B: It is not possible to preform requests over the same port, at the same time, but the threads just happens to use the same port at a slightly different time (until its not, at ~1000)

Or C: It is simply not allowed, after a bunch of request some network-overlord had enough and blocks new requests.

I must admit, I have limited networking knowledge to pinpoint the cause here or find a solution for that matter. What is the best way to approach a task like this? Any help is appreciated.

0

There are 0 best solutions below