i'm implementing a Transform stream which is piped by mongoose cursor (which works like readable stream, (or maybe it's real readable stream))
const myTransform = new Transform({
readableObjectMode: true,
writableObjectMode: true,
transform(chunk: Document, encoding, callback) {
this.push(chunk?.toObject() ?? null); // <- transforms data
callback(); // <- calls callback
},
});
MyMongooseModelWithHugeData.find()
.cursor()
.pipe(myTransform)
.pipe(someWriteStream)
Yup. My current code works fine.
But I found out the callback inside transform implementation, receives second parameter (which looks like classic nodejs async callback style obviously).
So, I changed my code like below, and found out that it also works fine..
const myTransform = new Transform({
readableObjectMode: true,
writableObjectMode: true,
transform(chunk: Document, encoding, callback) {
callback(null, chunk?.toObject() ?? null); // <- transforms data and calls callback.
},
});
And I felt somthing awkward. I've search some blogs about creating Transform stream. And all articles that I found was telling same that I have to use this.push() and call callback().
But none of them did even mention about the second parameter of callback().
So, my question is:
- Is there's difference between
this.push()andcallback()when passing transformed value? - Why many people recommend using
this.pushinstead ofcallback?
from nodejs documantation
https://github.com/nodejs/node/blob/master/doc/api/stream.md#transform_transformchunk-encoding-callback
It is possible that no output is generated from any given chunk of input data.
The
callbackfunction must be called only when the current chunk is completely consumed. The first argument passed to thecallbackmust be anErrorobject if an error occurred while processing the input ornullotherwise. If a second argument is passed to thecallback, it will be forwarded on to thetransform.push()method.In other words, the following are equivalent: