Intermittent failure (127) to find command in yarn

Question

I'm trying to build something on gcloud, specifically a web site with Yarn and Node.

This is the script I'm trying to run:

"build:langs": "printenv && ls -lah node_modules/.bin && BABEL_ENV=test babel-node scripts/translate.js",

Via

$ yarn run build:langs

Which outputs https://gist.github.com/haf/ebc623bfce5520432c136e44496b58fb — the interesting bits here (formatted for readability):

Step #2 - "js-build-langs": 
PATH=/workspace/src/site.com/node_modules/.bin
  :/usr/local/share/.config/yarn/link/node_modules/.bin
  :/workspace/src/
...
total 40K
Step #2 - "js-build-langs": drwxr-xr-x 2 root root 4.0K Jun 3 19:18 .
Step #2 - "js-build-langs": drwxr-xr-x 1194 root root 36K Jun 3 19:19 ..
Step #2 - "js-build-langs": lrwxrwxrwx 1 root root 20 Jun 3 19:18 JSONStream -> ../JSONStream/bin.js
Step #2 - "js-build-langs": lrwxrwxrwx 1 root root 19 Jun 3 19:18 _mocha -> ../mocha/bin/_mocha
...
Step #2 - "js-build-langs": lrwxrwxrwx 1 root root 30 Jun 3 19:18 babel-node -> ../babel-cli/bin/babel-node.js
...
Step #2 - "js-build-langs": /bin/sh: 1: babel-node: not found
Step #2 - "js-build-langs": error Command failed with exit code 127.
Step #2 - "js-build-langs": info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

An in container builder:

That's not all, because in this freak-show, a few minutes (now: hours) earlier :

The cloudbuild target looks like:

steps:
# JS from https://console.cloud.google.com/gcr/images/cloud-builders/GLOBAL/yarn?gcrImageListsize=50:
- name: gcr.io/cloud-builders/yarn:node-8.11.0
  waitFor: ["-"]
  id: js-install
  dir: src/site.com
  args: ["install"]

- name: gcr.io/cloud-builders/yarn:node-8.11.0
  waitFor: ["js-install"]
  id: js-build-prod
  dir: src/site.com
  args: ["run", "build:prod"]

- name: gcr.io/cloud-builders/yarn:node-8.11.0
  waitFor: ["js-build-prod"]
  id: js-build-langs
  dir: src/site.com
  args: ["run", "build:langs"]

And packages.json snip:

  "devDependencies": {
    "@babel/core": "^7.0.0-beta.49",
    "@babel/node": "^7.0.0-beta.49",

All in all, I'm very confounded. Why does it not work despite PATH containing the binary js file, which is executable non the less?? Why can it find webpack but not babel-node? Why can I run it locally even after a full git clean -fxd? And why would it work sometimes but not other times?

My gut is telling me this is some sort of race condition in GCB

Answer 1

EDIT 3: this comes up once in a while for us, and here's our current thoughts.

When we chain yarn-commands with && in package.json, that causes issues with creating e.g. node_modules/mocha/bin/mocha.js or other executable js files.

We made each command in packages.json non-chained and cleaned out node_modules. At least one build has gone through now.

Googling '127 command not found' and 'yarn' gives no hints whatsoever about this.

Answer 2

When you run on the hosted Google Container Builder, the only data that gets shared across build steps is data in the shared /workspace directory (and configured volumes, but you aren't using any in the snippet you provided).

I'm not an npm or yarn expert -- but have any of your packages changed their node_modules location in the package.json configuration? Note that yarn install takes a --modules-folder option, which may be helpful.