When use vm2 in worker_threads, is it possible to share a NodeVM instance between workers?

962 Views Asked by At

I am using worker_threads and vm2 to implement a serverless-like thing, but I cannot get a NodeVM instance in the main thread and then pass through workData(because of worker_threads's limitation), so I can only new NodeVM in a worker thread per request, inside which I cannot reuse a vm instance and the cost hurts.

The new NodeVM() takes 200 ~ 450 ms to finish, so I wish to pre-init a reusable instance.

const w = new Worker(`
    (async () => {
      const { workerData, parentPort } = require('worker_threads');
      const { NodeVM } = require('vm2');
      const t = Date.now();
      const vm = new NodeVM({ // cost 200 ~ 450 ms
        console: 'inherit',
        require: {
          external: [ 'request-promise', 'lodash' ],
          builtin: [],
          import: [ 'request-promise', 'lodash' ], // faster if added
        },
      });
      console.log('time cost on new NodeVM:', Date.now() - t);
      const fnn = vm.run(workerData.code, workerData.filename);
      console.log('time cost by initializing vm:', Date.now() - t);
      try {
        const ret = await fnn(workerData.params);

        parentPort.postMessage({
          data: typeof ret === 'string' ? ret : JSON.stringify(ret),
        });
      } catch (e) {
        parentPort.postMessage({
          err: e.toString(),
        });
      }
      console.log('----worker donex');
    })();
  `,
  {
    workerData: {
      params,
      code,
      dirname: __dirname,
      filename: `${__dirname}/faasVirtual/${fn}.js`,
    },
    eval: true,
  });

Can anybody give me some advice?

Thanks a lot.

1

There are 1 best solutions below

0
Yuefei Ma On

I've decided to prohibit external module import. Because require is internally readFileSync, which costs most of the time, and the http module within node itself can be used to replace request-promise.

After comment out external option, the average time cost for init is roughly 10+ms, which is acceptable for now.

But if worker_threads can clone function object through workerData, it would be more efficient.