Apache Commons Exec Problems Handling Regex Argument?

Question

I'm trying to use Apache Commons Exec to run a git command which uses a regex.

When I form my CommandLine and print it out it looks like this:

[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testFindByAdAccount).*", --, *Test.java]

However when I execute this, git returns no results, resulting in an exit code 1.

When I run this command manually though, it returns plenty of results and succeeds. Changing the --extended-regexp argument to just a string like testFindByAdAccount does yield results when run via Exec, so I think Apache Commons is doing something to the regexp argument making it invalid. Any ideas what is going on?

EDIT: Adding a reproducible example

1. Clone https://github.com/ragurney/min-example
2. Run gradlew shadowJar to produce jar file for project
3. Run the app with java -jar app/build/libs/app-all.jar
4. Note the output which shows the command printed fails with an exit code 1 (because there are no results returned by the git command)

$ java -jar app/build/libs/app-all.jar
HELLOOOOOO
WD:::  null
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testAppHasAGreeting)\(\).*", --, *Test.java]
WD:::  /Users/rgurney/Src/personal/min-example

Exception in thread "main" java.lang.RuntimeException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    at min.example.App.lambda$runCommand$1(App.java:74)
    at io.vavr.control.Try.getOrElseThrow(Try.java:748)

5. Running the command manually does produce expected results:

$ git --no-pager grep --line-number --untracked --extended-regexp "^\s*public void\s+(testAppHasAGreeting)\(\).*" -- "*Test.java"
app/src/test/java/min/example/AppTest.java:11:    public void testAppHasAGreeting() {

Answer 1

I got a clue as to what's going on here when the sample you provided worked just fine on my Windows laptop but failed on my Linux desktop.

Once I made sure the git version wasn't the culprit (tested several versions between 2.17 and 2.39 on both machines), I figured the difference must be in the way different shells handle quoting. Specifically, the only argument here that has any potential quoting issues is the regex ("^\s*public void\s+(testFindByAdAccount).*"), which is added to the command line by commandLine.addArgument(regex);.

addArgument may look innocuous, but under the hood, it allows the CommandLine to handle the quoting itself (i.e., addArgument(String argument) calls addArgument(String argument, true). Since you've handled the quoting yourself, you should not allow the CommandLine to handle the quoting, and should explicitly call it with the second argument false. i.e.:

public static List<String> grep(String regex, String filePattern, String wd) {
    CommandLine commandLine = CommandLine.parse("git");
    commandLine.addArgument("--no-pager");
    commandLine.addArgument("grep");
    commandLine.addArgument("--line-number");
    commandLine.addArgument("--untracked");
    commandLine.addArgument("--extended-regexp");
    commandLine.addArgument(regex, false); 
    // Here -----------------------^
    commandLine.addArgument("--");
    commandLine.addArgument(filePattern);

    System.out.println(commandLine);

    return List.of(runCommand(commandLine, wd).split("\n"));
}

This takes the quote-handling logic away and ensures the same code runs smoothly both on Windows and Linux (at least those I've tested).