Stream a raw pdf to client is taking to much time

Question

OS: ubuntu 20.04 Nodejs: 14.18.3 npm: 6.14.15 pdfMake: 0.1.72 (for create pdf files from a file or db queries)

Currently, I try make a pdf from a file (it could be a db query) and I want to send it to the client through stream. Send chunks of data to the users to avoid buffering the server's ram. I've already send it to the user, but it takes to much time to download a 300kb pdf, almost 15 secs. I've seem that every chunk received in the client is 16kb, that is the unchangeable (I think) sizes of the res object.

I use a readable stream coming from pdfMake library and try to pipe into the res object (express).

Here is the creation of the file (pdfMake library)

let bigfile = fs.readFileSync('./bigfile.txt', 'utf8');

function docDefinition() {
    let content = [{
        text: 'This is a header',
        alignment: 'center',
        fontSize: 25,
        margin: [0,0,0,0]
    }]
    
    content.push({ text: bigfile })

    let docDefinition = {
        content: content
    }

    return docDefinition
}

Here, I generate the row pdf itself "as a readable stream" and pass it to my express route in a callback

const generatePdf = (docDefinition, callback) => {

    try {
        const fontDescriptors = {
            Roboto: {
                normal: path.join(__dirname, '/fonts/Roboto-Regular.ttf'),
                bold: path.join(__dirname, '/fonts/Roboto-Medium.ttf'),
                italics: path.join(__dirname, '/fonts/Roboto-Italic.ttf'),
                bolditalics: path.join(__dirname, '/fonts/Roboto-MediumItalic.ttf'),
            },
        };
        const printer = new pdfMakePrinter(fontDescriptors);

        const doc = printer.createPdfKitDocument(docDefinition);

        /* something to add, **const = doc** has its own event for streaming (I guess)
        doc.on('data', chunk => {
           // send current chunk to client somehow
        })

        doc.on('end', chunk => {
           // finished the streaming
        })        
        */

        callback(doc)
    } catch (error) {
        throw error;        
    }
}

My main server.js, were I try to send the pdf to the user in a chuncked transafer way

app.get('/file', (req, res) => {
    generatePdf(
        docDefinition, 
        readable => {
            res.set({
                "Content-Type": "application/pdf",
                "Transfer-Encoding": "chunked",
                "Connection": "keep-alive"
            });

            console.log('res.HighWaterMark', res.writableHighWaterMark);
            console.log('readable highWaterMark', readable._readableState.highWaterMark);

            readable.pipe(res)
            readable.end()
        })
})

I just try to streaming a video in my browser and It was very easy cos the fs internal module. But here I using a external library (pdfmake) to make my readable stream. I'm very beginner with streams in node. I very appreciate any suggestion o help with that particular problem. :S

The source code could be fund here: https://github.com/biagiola/streamPDFtoClient

Answer 1

You don't need a Readable because Express' Response is based node's http.serverResponse which is a stream.

Edit: Just pass the respose to generatePdf and use doc.pipe(callback) and doc.end() there. I change callback to res in my sample code:

const pdfMakePrinter = require('pdfmake');

const generatePdf = (docDefinition, res) => {
  // ... setup code here.
  const doc = printer.createPdfKitDocument(docDefinition);
  doc.pipe(res);
  doc.end();
}

server.js:

app.get('/file', (req, res) => {
    generatePdf(
        docDefinition, 
        res);
});

Further Edits:

It could be the speed of the library and you may have to live with it. When I use the following code to time the generation process (removing the creating of raw document) I get about 7.25 seconds on my system.

app.get('/file', (req, res, next) => {

    res.attachment('myFancyPdf.pdf')
    const start = Date.now();
    generatePdfBase64(
        docDefinition,
        res);
    next(console.log(Date.now() - start))
});

My development system is an Intel i7-10700T CPU at with 32GB of RAM.