File streamed from node and made into blob on javascript client is much larger and corrupted

Question

I am working with the Clio API, a third party API that has some cloud based document storage. I would like our users to download video/audio documents from their API but, due to it's design, I have to use my node.js server as a proxy to get around CORS restrictions.

The flow is:

1. Users make a request to download document to my server
2. My server makes a request to the API to download the document
3. My server streams response to client

Because the documents may be large (1GB+), I am attempting to stream the files fetched by my server to the client.

The problem is that the files received on the client are corrupted and larger than the actual files. Here is how I am currently making the requests (note: we typically use axios but I am not sure how to do what I do below with that package, so for now I am using the request package for proof of concept):

For an MP4 file that is about 220kb in size:

request({
      url: `https://app.clio.com/api/v4/documents/${documentId}/download.json`,
      headers: { Authorization: `Bearer ${accessToken}` },
    }).pipe(res);

Then on the client we receive the response like:

response.blob()

The confounding thing is that the resulting blob is as such:

Blob {size: 407852, type: "application/mp4"}

The size is almost double the actual size of the file and I can not understand why. Furthermore, downloading and playing the mp4 from here is impossible because the mp4 file is corrupted and can't be played. This is especially strange because if I decide to pipe the server request to a file on my server like so:

    const filepath = path.resolve(__dirname, "test.mp4");
    const writer = fs.createWriteStream(filepath);

    request({
      url: `https://app.clio.com/api/v4/documents/${documentId}/download.json`,
      headers: { Authorization: `Bearer ${accessToken}` },
    }).pipe(writer);

The resulting "test.mp4" has the correct size and plays okay.

The issue seems to be with streaming the file to the client specifically. When I attempt to view the raw data that arrives on the client (via response.text()), a sample of what I see is the following:

\u0000\u0000\u0000\u001cftypmp42\u0000\u0000\u0000\u0001mp41mp42isom\u0000\u0000\u0000\u0001mdat\u0000\u0000\u0000\u0000\u0000\u0003UJ!*Te&(�B��4�p*hUī\u0012��Т�\u0012ɾ���0�P}�C$��}<��Gc�^�#~��ʠ6�^��m���<E���Y\u0013\t(Ѳ��x�\u0015/3c�(�����\u000b�4O\u0006�{$�d_J�\\j���^�ԑ\u0017~�&��iǠN�QV��\t\u0006%�B��05\u0012�kW�l\u0006hfZ�q7\u0000\u0006�v%��G���/2*\u000e\u0001-�^^6i�|

This appears to be UTF8 encoded data with a number of - I'm guessing - replacement characters. I think this may be what is inflating the size of the blob and corrupting the mp4 file.

What could be happening?

Answer 1

I was facing the same problem.I added below line to my axios configuration (client side). It solved my issue.

responseType: 'arraybuffer'