I am storing some json data as a gzipped file on a server (in order to reduce the amount of transported data, but also to reduce disk usage on the server).
Now, I want to decompress and parse the json data on the client side (browser running react/node).
I figured, using oboe.js and could be an appropriate choice for this task. Unfortunately, it seems impossible to directly pass oboe a stream when working in browser. The solution I have fallen back on is to feed chunks of data to oboe once they are unzipped (see code at the bottom of this post).
Though this does work, it freezes the browser for a while if the node listener takes some time to process the node (surely, it will take some time to parse the data, but I would like the app to stay responsive and possibly even update the view according to the already processed data).
I suspect all data is basically fed to oboe at once instead of waiting until the last chunk was read. Is this correct? Is there an appropriate way of slowing down the amount of data that is being fed to oboe, or is there a more direct way of working with a gzip-stream and oboe?
var o = oboe();
var gunzip = zlib.createGunzip();
gunzip.on('data', (chunk) => o.emit('data', chunk.toString()))
.on('end', o.emit('end'));
request(options).on('response', response => {
o.emit('start', response.statusCode, response.headers);
response.pipe(gunzip);
};
o.node('{name}', () => {
//Do stuff with node
return oboe.drop;
});
Seems you cant use correctly the pipe functionality (like speed adapting) due to "wrapper" in lines 4-5 Just remove it.
This more information about piping node.js