Question about NginX proxy, is there a way to control the chunk size (ie as to mimic the chunks being sent back by the proxy-ed backend server, Not breaking chunks)
I have a backend server that is built to return chunked responses to clients where each chunk is set to be a full audio part.
When running behind an NginX proxy server, the NginX seems to be breaking the full chunks returned by the backend server into smaller chunks. All data is sent properly, but the original chunks returned by the server are broken into several smaller chunks
Here is a sample output,
2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 1414 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 11360 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2840 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2840 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2840 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2840 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 704 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2834 bytes 2017/09/11 17:16:20 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 7100 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 9940 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 4260 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 1420 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 2019 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 4254 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 14200 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 1398 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 379 bytes 2017/09/11 17:16:21 DEBUG - com.nuance.httpservlet.utils.HttpTtsParser: readChunk 0 bytes
From server side logs 7 capture, the first 22838 bytes (1414 + 11360 + 840 +2840 +2840 + 2840 + 704) were originally sent as a single chunk by the back end server,
I have played with the proxy_buffer_size, proxy_buffering and proxy_busy_buffers_size, options but i cannot find a way to force the NginX to maintain the same chunks as the backend (even with Huge buffer sizes and numbers), here is the setting i used fro that test (Not optimal, i know but explicitly generous to help narrow down the issue)
proxy_buffer_size 5m
proxy_buffering: 100 5m
proxy_busy_buffers_size: 5m
I scanned through docs and forums but i could not find a way to force NginX to keep the chunks as being returned by the server.
I see there is a way to disable chunked_transfer_encoding But this is NOT what i want, what i want is to keep chunked_transfer_encoding and control the NginX to mimic the same chunking as the original backend
Please advise if you have any tips.
Thank you
B
SO after looking at nginx code and starting a thread with the support e-mail group (but not answer yet)
we found that HAProxy has this very nice feature, which when we tested with, solved out issue....
option http-no-delay no option http-no-delay Instruct the system to favor low interactive delays over performance in HTTP
the network optimizations which favor performance by instructing the system to wait for enough data to be available in order to only send full packets. Typical delays are around 200 ms per round trip.
we run with HAproxy with this option disabled....so the chunks were read properly and forwarded properly