Why does writing to HTTP stream pause for >100ms every few writes?

Question

I am trying to stream MJPEG video over http using a multipart stream with aiohttp.

I have an implementation based on this Gist by james4388. I am trying to send a new JPEG image every 30ms or so currently. Timing the loop shows that the await mpwriter.write(response, close_boundary=False) often takes 0.3ms but every 3-5 times takes 150-200ms. This limits the maximum frame rate I can achieve and introduces latency into the stream which I am trying to avoid.

The loop used is

while True:
        _, frame = wc.read()
        if frame is None:
            continue
        with MultipartWriter('image/jpeg', boundary=boundary) as mpwriter:
            result, encimg = cv2.imencode('.jpg', frame, encode_param)
            data = encimg.tostring()
            mpwriter.append(data, {
                'Content-Type': 'image/jpeg'
            })
            await mpwriter.write(response, close_boundary=False) # 'Pauses' here once every 3-5 times
        await response.drain()

I have also tried this using BaseHttpServer. This exhibits the same behaviour when calling self.wfile.write(jpg.tostring()).

I assume this pause happens when an underlying buffer is flushed out over the network, although I call response.drain() after each part. Each JPEG frame (so each part of the multi-part stream) is 441kB in size, so for 30fps I would be sending 13MB/s but currently I am achieving 5.3MB/s. This is on dedicated test LAN connection so I don't think bandwidth is an issue (Another appliance using MJPEG streaming achieves >15MB/s).

Making the image size smaller (e.g. with higher compression) increases the number of frames I can send before hitting the block, but the overall block is still there and my throughput is still only 4.3MB/s.