pyCurl spikes CPU when I add option MAX_SEND_SPEED_LARGE

Question

My app uses PyCurl to upload multiple files one by one. It works all good with all cURL options and CPU consumption remains below 1%. When I add MAX_SEND_SPEED_LARGE option to throttle bandwidth, CPU consumption goes to 100%.

c.setopt(c.MAX_SEND_SPEED_LARGE, int(rate_limit))

Further investigation reveals, it only happens on OS X 10.9

Answer 1

• Are you working with libcurl < 7.31.0?
• Also: are you using the progress function?

If yes, I would say you may have hit a bug that has been fixed in libcurl 7.31.0, i.e curl_easy_perform: avoid busy-looping:

I am using pycurl in a small script to download coursera videos [...] it hangs 1 core of the machine all the time [...] I can confirm that this occurs, and only when rate limiting for upload/download is in place.

The patch can be seen here.