Streaming Results from Mochiweb

Question

I have written a web-service using Erlang and Mochiweb. The web service returns a lot of results and takes some time to finish the computation. I'd like to return results as soon as the program finds it, instead of returning them when it found them all.

edit:

i found that i can use a chunked request to stream result, but seems that i can't find a way to close the connection. so any idea on how to close a mochiweb request?

Answer 1

To stream data of yet unknown size with HTTP 1.1 you can use HTPP chunked transfer encoding. In this encoding each chunk of data prepended by its size in hexadecimal. Last chunk is a zero-length chunk, with the chunk size coded as 0, but without any data.

If client doesn't support HTTP 1.1 server can send data as binary chunks and close connection at the end of the stream.

In MochiWeb it's all works as following:

1. HTTP response should be started with Response = Request:respond({Code, ResponseHeaders, chunked}) function. (By the way, look at the code comments);
2. Then chunks can be send to client with Response:write_chunk(Data) function. To indicate client the end of the stream chunk of zero length should be sent: Response:write_chunk(<<>>).
3. When handling of current request is over MochiWeb decides should connection be closed or can be reused by HTTP persistent connection.