Issue in blob storage to fileShare big file transfer : Using fileRange (nodejs Cloud function). It transferring partial files

Question

Issue in blob storage to fileShare big file transfer : Using fileRange (nodejs Cloud function). It transferring partial files.

When we transfer file of size 10MB - it transfers only 9.7MB When we transfer file of size 50MB - it transfers only 49.5MB

It gives issues that: Stack: RangeError: contentLength must be > 0 and <= 4194304 bytes

Code snnipet:

const fileName = path.basename('master/test/myTestXml.xml')
const fileClient = directoryClient.getFileClient(fileName);
const fileContent = await streamToString(downloadBlockBlobResponse.readableStreamBody)
await fileClient.uploadRange(fileContent, 0,fileContent.length,{
    rangeSize: 50 * 1024 * 1024, // 4MB range size
    parallelism: 20, // 20 concurrency
    onProgress: (ev) => console.log(ev)
  });

After transferring partial file it give error - any suggestion: how can we transfer big files using rangeSize.
Stack: RangeError: contentLength must be > 0 and <= 4194304 bytes

Answer 1

If you just want to transfer some files from Azure blob storage to Azure File share, you can generate a blob URL with SAS token on your server-side and use the startCopyFromURL function to let your File share copy this file instead of downloading from Azure blob storage and upload to file share. It eases the pressure of your server and the copy process is quick as it uses Azure's internal network.

Just try code below:

const { ShareServiceClient } = require("@azure/storage-file-share");
const storage = require('azure-storage');

const connStr = "";
const shareName = "";
const sharePath = "";

const srcBlobContainer ="";
const srcBlob="";

const blobService = storage.createBlobService(connStr);

// Create a SAS token that expires in 1 hour
// Set start time to five minutes ago to avoid clock skew.
var startDate = new Date();
startDate.setMinutes(startDate.getMinutes() - 5);
var expiryDate = new Date(startDate);
expiryDate.setMinutes(startDate.getMinutes() + 60);

//grant read permission
permissions = storage.BlobUtilities.SharedAccessPermissions.READ;

var sharedAccessPolicy = {
    AccessPolicy: {
        Permissions: permissions,
        Start: startDate,
        Expiry: expiryDate
    }
};

var srcBlobURL = blobService.getUrl(srcBlobContainer,srcBlob)
var sasToken = blobService.generateSharedAccessSignature(srcBlobContainer, srcBlob, sharedAccessPolicy)
var srcCopyURL = srcBlobURL + "?" + sasToken

const serviceClient = ShareServiceClient.fromConnectionString(connStr);
const fileClient = serviceClient.getShareClient(shareName).getDirectoryClient(sharePath).getFileClient(srcBlob);

fileClient.startCopyFromURL(srcCopyURL).then(function(){console.log("done")})

I have tested on my side in my storage account, result:

I copy a file with about 11 MB, it spends about 5s.