Papa parser to await for step

Question

I was able to configure all in papa parser and it works well even with 200k items parsing. So I can process all the data and them await for each row in the result array. the problem is if I am using a step how I can make the step await the previous step? This is causing the parser to fires thousands of calls and then blocking me. I tried to wrapped the step in another function but that did not help either.

here is a sample of what I am doing in the step call

async function stepFn(results, parser) {
stepped++;
      (await (async function () {
        if (results) {
          if (results.data) {
            rowCount += results.data.length;
            console.log('checking if item exists');
            var itemExistsP = await restcallToRetrieve();
            if (itemExistsP.length > 0) {
              console.log('Item exists, skipping');
              //return;
            } else {
              console.log('adding item to list');
              await restcalltoAdd();
              // return item;
            }
          }
          if (results.errors) {
            errorCount += results.errors.length;
            firstError = firstError || results.errors[0];
          }
        }
      }()));
}

Here is the Parser call:

Papa.parse($('#parse-file')[0].files[0], {
            delimiter: '',
            header: true,
            dynamicTyping: false,
            skipEmptyLines: true,
            preview: 0,
            encoding: '',
            step: stepFn,
            worker: false,
            comments: '',
            download: false,
            complete: completeFn,
            error: errorFn,
            });

Answer 1

Just create an async stepFunction:

async function stepFn(results, parser) {
    parser.pause(); // pause the parser

    if (results.data) {
        rowCount += results.data.length;

        let itemExists = await restCallToRetrieve(/*probably the data goes here*/);
        if (itemExists.length > 0) {
            console.log('Item exists, skipping');
        }
        else {
            console.log('adding item to list');
            await restCallToAdd(/*probably the data goes here*/);
        }
    }
    else if (results.errors.length > 0) { //Note that I am checking if there are errors
        errorCount += results.errors.length;
        firstError = results.errors[0];
    }
    parser.resume(); // resume the parser
}

Then in the parse config object:

Papa.parse(file, {
    step: stepFn
});

It would also be good to put everything you await in a try catch block but that's of course beside the point.