Procesing big excel file with nodejs in SAP S/4Hana Application

Question

I would like to ask for some help with solving the problem I'm facing (not a javascript developer, bare in mind. ). Any help would be grateful.

• Process: User uploads selected excel file from UI. In back-end we process file with xlsx/exceljs javascirpt libraries (tried both - same end result) to populate custom data structures for later uses.

• Problem: with big excel files (from 10k rows and more, with lots of attributes) I get following error:

  ...Error: Corrupted zip: can't find end of central directory.

• Additional info: by running the same code on plain nodejs server (locally) - it works. On SAP server - it doesn't. With small files - all good.

• Tech: SAP S/4Hana, OData v4, xlsx v0.18.5, exceljs v4.3.0, node >=v12.

• Code example:

   //with exceljs library
   fromXlsx_1: async (buffer, cols) => {
  
   const workbook = new ExcelJS.Workbook();
   await workbook.xlsx.load(buffer);//<-- error
  
   workbook.worksheets.forEach(function(sheet) {
  
       //some code
  
       });
  
  },
  //with xlsx library
  fromXlsx_2: (buffer, cols) => {
       var book = XLSX.read(buffer, { type: 'buffer', cellDates: true, cellText: false);//<-- error
       var sheet = book.Sheets[book.SheetNames[0]];
       const result = XLSX.utils.sheet_to_json(sheet, { header: Object.entries(cols).map(a => a[0]), raw: true });
  
       //some code
  }
  

If more information is needed- ask away. Thank you for your time.

Answer 1

Managed to solve it. The root cause were - lack of knowledge and the implementation of package "busboy" (made a presumption, that colleagues, that wrote the custom package to "process" files, did not make a mistake). If you stumble on similar problem like me, double check the "busboy" implementation, if its parsing the file correctly and it WAITS for the the file to be processed all - with async/await concept.