Moving from Yarn to Npm in Rails project breaks Webpacker with missing dependencies errors

Question

I am migrating a Rails project from Yarn to Npm because my team doesn't plan on upgrading to Yarn Berry. I've removed all traces of Yarn and replaced them with Npm and followed this Rails/Webpacker/Npm tutorial.

When starting my webpack server with npm run webpack-dev-server however, I run into the following errors:

Module not found: Error: Can't resolve 'worker_threads' in './node_modules/terser-webpack-plugin/node_modules/jest-worker/build/workers'

Not sure what could be causing this error. I've deleted yarn.lock and the node_modules directory multiple times to no avail.

Here is my package.json

{
  "dependencies": {
    "@hapi/hoek": "^8.5.1",
    "@rails/webpacker": "^5.2.1",
    "autoprefixer": "^8.0.0",
    "axios": "^0.18.1",
    "babel-core": "^6.26.0",
    "babel-loader": "7.x",
    "babel-plugin-syntax-dynamic-import": "^6.18.0",
    "babel-plugin-transform-class-properties": "^6.24.1",
    "babel-plugin-transform-object-rest-spread": "^6.26.0",
    "babel-polyfill": "^6.26.0",
    "babel-preset-env": "^1.6.1",
    "camelize2": "^1.0.0",
    "coffee-loader": "^0.9.0",
    "coffeescript": "^1.12.7",
    "compression-webpack-plugin": "^3.0.1",
    "consolidate": "^0.16.0",
    "css-loader": "^3.4.1",
    "file-loader": "^4.3.0",
    "glob": "^7.1.2",
    "got": "^8.0.1",
    "handlebars": "^4.5.2",
    "jest-serializer-vue": "^2.0.2",
    "js-yaml": "^3.13.1",
    "mem": "^4.0.0",
    "mini-css-extract-plugin": "^0.9.0",
    "moment": "^2.19.4",
    "moment-timezone": "^0.5.27",
    "node-gyp": "^4.0.0",
    "node-sass": "^4.14.1",
    "path-complete-extname": "^0.1.0",
    "postcss-import": "^12.0.1",
    "postcss-loader": "^3.0.0",
    "precss": "^2.0.0",
    "rails-erb-loader": "^5.5.2",
    "resolve-url-loader": "^3.1.1",
    "sass-loader": "^8.0.0",
    "style-loader": "^1.1.2",
    "terser-webpack-plugin": "^2.3.1",
    "vue": "^2.5.9",
    "vue-loader": "^15.9.3",
    "vue-template-compiler": "^2.5.9",
    "webpack-manifest-plugin": "^2.2.0",
    "webpack-merge": "^4.1.1",
    "whiskers": "^0.4.0"
  },
  "devDependencies": {
    "@babel/core": "^7.0.0",
    "@babel/plugin-proposal-object-rest-spread": "^7.8.3",
    "@babel/polyfill": "^7.8.3",
    "@vue/test-utils": "^1.0.0-beta.30",
    "babel-jest": "^21.2.0",
    "jest": "^26.0.1",
    "npm-install-webpack-plugin": "^4.0.5",
    "vue-jest": "^3.0.5",
    "webpack": "^4.44.2",
    "webpack-cli": "^3.3.12",
    "webpack-dev-server": "^3.11.0"
  },
  "engines": {
    "node": "12.18.4"
  },
  "scripts": {
    "webpack-dev-server": "webpack-dev-server",
    "webpack": "webpack"
  }
}

Here is my bin/webpack-dev-server script

#!/usr/bin/env ruby
$stdout.sync = true

require "shellwords"
require "yaml"

ENV["RAILS_ENV"] ||= "development"
RAILS_ENV = ENV["RAILS_ENV"]

ENV["NODE_ENV"] ||= RAILS_ENV
NODE_ENV = ENV["NODE_ENV"]

APP_PATH          = File.expand_path("../", __dir__)
CONFIG_FILE       = File.join(APP_PATH, "config/webpacker.yml")
NODE_MODULES_PATH = File.join(APP_PATH, "node_modules")
WEBPACK_CONFIG    = File.join(APP_PATH, "config/webpack/development.js")

def args(key)
  index = ARGV.index(key)
  index ? ARGV[index + 1] : nil
end

begin
  dev_server = YAML.load_file(CONFIG_FILE)["development"]["dev_server"]

  protocol = args('--https') || dev_server["https"] ? 'https' : 'http'
  host = args('--host') || dev_server["host"]
  port = args('--port') || dev_server["port"]

  DEV_SERVER_HOST = "#{protocol}://#{host}:#{port}".freeze
rescue Errno::ENOENT, NoMethodError
  puts "Webpack dev_server configuration not found in #{CONFIG_FILE}."
  puts "Please run bundle exec rails webpacker:install to install webpacker"
  exit!
end

newenv = {
  "NODE_PATH" => NODE_MODULES_PATH.shellescape,
  "ASSET_HOST" => DEV_SERVER_HOST.shellescape
}.freeze

cmdline = ["npm", "run", "webpack-dev-server", "--color", "--config", WEBPACK_CONFIG] + ARGV

Dir.chdir(APP_PATH) do
  exec newenv, *cmdline
end

Let me know if more code snippets would be helpful to help debug!

Thanks