readable stream on data event losing first/last character from chunk

Question

I am using ssh2-sftp-client for nodeJS to connect to sftp server and get a file But I've meet a problem, when are more chunks than one, the data is not received correctly, it loses one character between chunks:
ex.: file on sftp has 'some string from sftp file'
and if it is splitted in chunks, the received data will be like:
first chunk : 'some string f'
second chink: 'om sftp file'
in this example 'r' is lost

const getFile = readable => new Promise((resolve, reject) => {
  let file = '';
   readable.on('data', (chunk) => { file += chunk; }); 
  readable.on('end', () => resolve(file));
  readable.on('error', reject);
});

const readable = await sftp.get(fileName, false);

sftp.get() return NodeJS.ReadableStream

Does someone meet same problem?

Answer 1

After a long research I found the problem, in module sftp-stream, for readable stream highWaterMark is seted to 64*1024, and the bug is that if a chunk has 64*1024 bites then one bite is lost. And I just setted watermark to 64*1024-1 .

Answer 2

Is it more advisable to store chunks as a array and then concat them into a buffer and get the string value on end:

const fs = require('fs');

const getFile = readable => new Promise((resolve, reject) => {
  const file = [];
  readable.on('data', chunk => file.push(Buffer.from(chunk, 'base64')));
  readable.on('end', () => resolve(Buffer.concat(file).toString()));
  readable.on('error', reject);
});


const readable = fs.createReadStream('package.json');

getFile(readable).then(file => console.log('file', file));

Runned it locally and getting the content of my file as expected.