Wildcard expansion (globbing) in a string composed of quoted and unquoted parts

Question

I have a line in a shell script that looks like this:

java -jar "$dir/"*.jar

, since I just want to execute whatever the jar file happens to be named in that folder. But this is not working as I expected. I get the error message:

Error: Unable to access jarfile [folder-name]/*.jar

It is taking the '*' character literally, instead of doing the replacement that I want. How do I fix this?

EDIT: It is working now. I just had the wrong folder prefix :/ For anybody wondering, this is the correct way to do it.

Answer 1

You just need to set failglob:

shopt -s failglob

to avoid showing literal *.jar when none are matched in a given folder.

PS: This will generate an error when it fails to match any *.jar as:

-bash: no match: *.jar

Answer 2

Explanation and background information

The OP's problem was NOT with globbing per se - for a glob (pattern) to work, the special pattern characters such as * must be unquoted, which works even in strings that are partly single- or double-quoted, as the OP correctly did in his question:

"$dir/"*.jar # OK, because `*` is unquoted

Rather, the problem was bash's - somewhat surprising - default behavior of leaving a pattern unexpanded (leaving it as is), if it happens not to match anything, effectively resulting in a string that does NOT represent any actual filesystem items.

• In the case at hand, "$dir" happened to expand to a directory that did not contain *.jar files, and thus the resulting string passed to java ended in literal *.jar ('<value of $dir>/*.jar'), which due to not referring to actual .jar files, resulted in the error quoted in the question.

Shell options govern globbing (more formally called pathname expansion):

• set -f (shopt -so noglob) turns off globbing altogether, so that unquoted strings (characters) that would normally cause a string to be treated as a glob are treated as literals.
• shopt -s nullglob alters the default behavior to expanding a non-matching glob to an empty string.
• shopt -s failglob alters the default behavior to reporting an error and setting the exit code to 1 in the case of a non-matching glob, without even executing the command at hand - see below for pitfalls.
• There are other globbing-related options not relevant to this discussion - to see a list of all of them, run { shopt -o; shopt; } | fgrep glob. For a description, search by their names in man bash.

---

Robust globbing solutions

Note: Setting shell options globally affects the current shell, which is problematic, as third-party code usually makes the - reasonable - assumption that defaults are in effect. Thus, it is good practice to only change shell options temporarily (change, perform action, restore) or to localize changing their effect by using a subshell ((...)).

---

shopt -s nullglob

• Useful for enumerating matches in a loop with for - it ensures that the loop is never entered, if there are no matches:

shopt -s nullglob # expand non-matching globs to empty string
for f in "$dir/"*.jar; do
  # If the glob matched nothing, we never get here.
  # !! Without `nullglob`, the loop would be entered _once_, with 
  # !! '<value of $dir>/*.jar'.
done

• Problematic when used with arguments to commands that have default behavior in the absence of filename arguments, as it can result in unexpected behavior:

shopt -s nullglob # expand non-matching globs to empty string
wc -c "$dir/"*.jar # !! If no matches, expands to just `wc -c`

If the glob doesn't match anything, just wc -c is executed, which does not fail, and instead starts reading stdin input (when run interactively, this will simply wait for interactive input lines until terminated with Ctrl-D).

---

shopt -s failglob

• Useful for reporting a specific error message, especially when combined with set -e to cause automatic aborting of a script in case a glob matches nothing:

set -e  # abort automatically in case of error
shopt -s failglob # report error if a glob matches nothing
java -jar "$dir/"*.jar  # script aborts, if this glob doesn't match anything

• Problematic when needing to know the specific cause of an error and when combined with the || <command in case of failure> idiom:

shopt -s failglob # report error if a glob matches nothing
# !! DOES NOT WORK AS EXPECTED.
java -jar "$dir/"*.jar || { echo 'No *.jar files found.' >&2; exit 1; }
# !! We ALWAYS get here (but exit code will be 1, if glob didn't match anything).

Since, with failglob on, bash never even executes the command at hand if globbing fails, the || clause is also not executed, and overall execution continues.

While the failed glob will cause the exit code to be set to 1, you won't be able to distinguish between failure due to a non-matching glob vs. failure reported by the command (after successful globbing).

---

Alternative solution without changing shell options:

With a little more effort, you can do your own checking for non-matching globs:

Ad-hoc:

glob="$dir/*.jar"
[[ -n $(shopt -s nullglob; echo $glob) ]] || 
  { echo 'No *.jar files found.' >&2; exit 1; }
java -jar $glob

$(shopt -s nullglob; echo $glob) sets nullglob and then expands the glob with echo, so that the subshell either returns matching filenames or, if nothing matches, an empty string; that output, thanks to command substitution ($(...)), is passed to -n, which tests whether the string is empty, so that the overall [[ ... ]] conditional's exit code reflects if something matched (exit code 0) or not (exit code 1).

Note that any command inside a command substitution runs in a subshell, which ensures that the effect of shopt -s nullglob only applies to that very subshell and thus doesn't alter global state.

Also note how the entire right-hand side in the variable assignment glob="$dir/*.jar" is double-quoted, to illustrate the point that quoting with respect to globbing matters when a variable is referenced later, not when it is defined. The unquoted references to $glob later ensure that the entire string is interpreted as a glob.

With a small helper function:

# Define simple helper function.
exists() { [[ -e $1 ]]; }

glob="$dir/*.jar"
exists $glob || { echo 'No *.jar files found.' >&2; exit 1; }
java -jar $glob

The helper function takes advantage of shell applying globbing upon invocation of the function, and passing the results of globbing (pathname expansion) as arguments. The function then simply tests whether the 1st resulting argument (if any), refers to an existing item, and sets the exit code accordingly (this will work regardless of whether nullglob happens to be in effect or not).

Answer 3

Make sure to have bash shell installed to use the command shopt described by @mklement0