How to use webpack with a monorepo (yarnpkg workspaces)

Question

I'm using yarn workspaces where the root directory has a package directory with all my repos. Each repo has its own node_modules directory containing its dependencies. The root node_modules directory contains all the dev dependencies for the whole project as well as all other dev related things such as webpack.config files. Webpack uses hot module reload for the express server package.

The problem I have is, how to configure webpack externals to exclude all node_modules directories through the whole project, not just in the root?

webpack-node-externals doesn't seem to work given this scenario.

Error message:

WARNING in ./packages/servers/express/node_modules/colors/lib/colors.js
127:29-43 Critical dependency: the request of a dependency is an expression

WARNING in ./packages/servers/express/node_modules/express/lib/view.js
79:29-41 Critical dependency: the request of a dependency is an expression

Webpack config:

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');
const StartServerPlugin = require('start-server-webpack-plugin');

module.exports = {
  entry: [
    'babel-polyfill',
    'webpack/hot/poll?1000',
    path.join(__dirname, '../packages/servers/express/server/index.js')
  ],
  watch: true,
  target: 'node',
  externals: [
    nodeExternals({
      whitelist: ['webpack/hot/poll?1000']
    })
  ],
  resolve: {
    alias: {
      handlebars: 'handlebars/dist/handlebars.js'
    }
  },
  module: {
    rules: [
      {
        test: /\.js?$/,
        use: 'babel-loader',
        exclude: /node_modules/
      }
    ]
  },
  plugins: [
    new StartServerPlugin('server.js'),
    new webpack.NamedModulesPlugin(),
    new webpack.HotModuleReplacementPlugin(),
    new webpack.NoEmitOnErrorsPlugin(),
    new webpack.DefinePlugin({
      'process.env': { BUILD_TARGET: JSON.stringify('server') }
    })
  ],
  output: {
    path: path.join(__dirname, '../packages/servers/express/.build'),
    filename: 'server.js'
  }
};

Answer 1

Yarn workspaces hoist compatible modules to the root node_modules directory leaving any incompatible (different semver, etc.) modules with the dependent workspace's node_modules directory. If a package is requested without using a relative path it is either native, from node_module's, or possibly a symlinked package from one of your workspaces. You probably want all of those packages to be external.

how to configure webpack externals to exclude all node_modules directories through the whole project, not just in the root?

I would try using a function with webpack's external option. You are passed the context of the require, the name of the module requested, and a callback to indicate whether this particular import (require) should be considered external.

externals: [
    (ctx, req, cb) => {
        if (!/node_modules/.test(ctx) && req[0] !== '.') {
            // Assumes you have defined an "entries" variable
            let notAnEntry = (path) => {
                return Object.keys(entries).every((entry) => {
                    return entries[entry] !== path
                });
            };

            if (notAnEntry(require.resolve(req))) {
                // This module is external in a commonjs context
                return cb(null, `commonjs ${req}`);
            }
        }

        cb();
    }
]

Answer 2

Thanks to @blackxored I was able to fix it on my project.

In your webpack config file do the following:

import nodeExternals from 'webpack-node-externals'

Then add

externals: [
  nodeExternals({
    modulesFromFile: true,
  }),
],

Answer 3

If using yarn workspaces with webpack-node-externals a better solution than setting modulesFromFile: true is to use the following externals setting in your webpack config:

externals: [
  nodeExternals(),
  nodeExternals({
    modulesDir: path.resolve(__dirname, 'path/to/root/node_modules'),
  }),
],

Essentially using two instances of nodeExternals. 1 for the package node_modules and one for the root node_modules.