Jetty HttpClient crashes on POST when using streams with available() method returning relatively small values

Question

I use Jetty HttpClient to send POST request with body around few MB. I want Jetty to start streaming the request as soon as possible, so I use setRequestContentSource method.

The problem is that when I use any input stream with available() method returning relatively small value (like 4096) Jetty sometimes crash with following error:

org.eclipse.jetty.io.EofException
    at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:911)
    at org.eclipse.jetty.client.HttpConnection.handle(HttpConnection.java:241)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:520)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:528)
    at java.lang.Thread.run(Thread.java:680)
Caused by: 
java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcher.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:100)
    at sun.nio.ch.IOUtil.write(IOUtil.java:71)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
    at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:195)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:285)
    at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:316)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:267)
    at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:846)
    at org.eclipse.jetty.client.HttpConnection.handle(HttpConnection.java:241)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:520)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:528)
    at java.lang.Thread.run(Thread.java:680)

It's non deterministic and it seems that putting Thread.sleep(10) in stream's read() method fixes the problem. This error can be also fixed when piped streams are used. Those 3 things make me think that this is some kind of race condition.

I suppose that this is bug in Jetty, but I want to be sure if I'm not doing anything weird in such scenario.

Answer 1

I get this a lot especially whilst going through a proxy and doing a multi part.

Buffer okBuffer = new ByteArrayBuffer(IOUtils.toByteArray(streamToTranslate));
exchange.setRequestContent(okBuffer);

I know this doesn't stream it, which is what you want to do, but for me there is a cap on the files I am posting so it works.