If my variable contains quoted globs and I expand it without using double quotes, why do the globs disappear?

I was trying to use unquoted strings expansions to pass two arguments to tar; the first is the command-line flag --exclude and the second contains a * character. In an attempt to avoid premature globbing, I tried quoting the *:

shopt -s nullglob
x="--exclude '*'"
echo tar $x

To my surprise, the '*' completely disappeared! Here was the output:

tar --exclude

I understand that I could switch from POSIX shell to Bash and use arrays to avoid this headache. But I still wonder: what the hell was going on with that POSIX shell snippet? Why did it delete the quoted globs? I would have expected that it would leave the quoted part alone, or go all the way and expand the glob. It did neither of those.

Asked By: hugomg

||

First of all, you are already using bash – or at least some non-POSIX options. (Neither shopt nor nullglob is POSIX.)

Now let’s explain what happened.

x="--exclude '*'"
echo tar $x

The variable $x is word-split on whitespace, and the resulting parts given over for globbing. --exclude has no wildcard and is left verbatim. '*' has no match (unless you have a file name literally starting and ending with a single quote), so because it contains a wildcard and you’ve set nullglob it is removed. The result is

tar --exclude

Since you are already using bash you might as well do it properly,

x=('--exclude' '*')
echo tar "${x[@]}"

If you want a POSIX solution the best I can offer is to reuse the main argument list:

set -- '--exclude' '*'
echo tar "$@"
Answered By: Chris Davies

For corner cases like this the order of shell expansions in the Bash Documentation is usually the cause of unexpected behavior. Posix has more-or-less the same behavior but stated less clearly.

Here’s what’s going on:

You’ve turn nullglob so globs that fail are empty.

Next you set a variable containing a literal asterisk, and single quotes.

You then echo the contents of the variable and the following things happen:

  1. The variable is expanded into text. That text contains a '*'
  2. The '*' is substituted for any file names present, in spite of the single quotes that you’d think would make it a string literal only. Nullglob turns this into an empty string
  3. The command is executed
Answered By: davolfman
Categories: Answers Tags:
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.