How can I have a Bazel genrule affect the original source tree?

Question

I have two Bazel genrules: one that creates a node_modules.tar file, and one that explodes the tar into actual files.

When I run the build though with bazel build //node_modules_ls I find that it explodes the tar into another environment in bazel-out, a symlink to somewhere else. If I add local=True, it will affect my actual source code tree.

Is this the intended use for local? Is there a better way to accomplish what I want here? I actually do want to affect my source code tree, because I haven't had time to adopt Bazel for all my build scripts.

genrule(
  name = "node_modules_ls",
  local = True, # Added this line to fix my issue!
  srcs = [
    ":node_modules",
  ],
  cmd = " && ".join([
    "tar xf $(location :node_modules)",
    "tar tf $(location :node_modules) > $@",
  ]),
  outs = [
    "out.txt",
  ],
)

genrule(
  name = "node_modules",
  srcs = [
    "package.json",
    "bun.lockb",
  ],
  cmd = " && ".join([
    "(cd packages/api && bun install --frozen-lockfile)",
    "tar cf $(location :node_modules.tar) packages/api/node_modules/",
  ]),
  outs = [
    "node_modules.tar",
  ],
)

Answer 1

Meaning of local is described in the docs and affect strategy used:

If set to True, this option forces this genrule to run using the "local" strategy, which means no remote execution, no sandboxing, no persistent workers.

Even it may perform actions locally, you can still only consume (in subsequent steps) declared outputs produced by the rule (actions) regardless of execution strategy used. If you wanted to change the tree, it'd be too late (and also wrong) at this point as build tree has been already loaded (read) and analyzed before executing on its actions.

For building up the tree / bringing content in, the mechanics would be to prepare and include the tree as external dependencies with (custom) repository_rule (or a bzlmod).

---

Looking at your snippet, I presume the bit that you're really asking about is the content dumped by tar xf $(location :node_modules), without it, to just get the (simplified by using static content), it's just a matter of simple srcs/outs:

genrule(
    name = "node_modules",
    srcs = [
        "a",
        "b",
    ],
    outs = ["node_modules.tar"],
    cmd = "tar chf $@ $(SRCS)",
)

genrule(
    name = "node_modules_ls",
    srcs = [":node_modules"],
    outs = ["out.txt"],
    cmd = "tar tf $< > $@",
)

(Note: there are actually already existing and portable rules ready for reuse that can create tarballs under rules_pkg.)

To explode a tarball and use bits therefrom... apart from repository rule, you can also define a custom rule which can return a declare and provide a directory with an exploded tree accessible to other actions.

---

One more edit. It appears the genrule can actually produce a directory output too. I haven't double checked how reliable directories are in this case (generally they can pose challenges).

So in theory, you may be able to expose the exploded content as a usable label:

genrule(
    name = "node_modules_unpacked",
    srcs = [":node_modules"],
    outs = ["unpacked"],
    cmd = "mkdir $@ && tar xf $< -C $@",
) 

and just to verify we got the stuff we wanted when referring it:

genrule(
    name = "node_modules_unpacked_ls",
    srcs = [":node_modules_unpacked"],
    outs = ["ls.txt"],
    cmd = "find $</ > $@",
)

Answer 2

In Bazel, you should not write to the source tree in general, more on that here:

https://www.aspect.dev/blog/bazel-can-write-to-the-source-folder

If you really need to, like in your case, to move an artifact for further consumption by other systems, you can use the AspectBuild rule that does exactly this:

https://github.com/aspect-build/bazel-lib/blob/main/docs/write_source_files.md