Bash: Why is eval and shift used in a script that parses command line arguments?
As I was looking this answer https://stackoverflow.com/a/11065196/4706711 in order to figure out on how to use parameters like
-s some questions rised regarding the answer’s script :
#!/bin/bash TEMP=`getopt -o ab:c:: --long a-long,b-long:,c-long:: -n 'example.bash' -- "$@"` if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi # Note the quotes around `$TEMP': they are essential! eval set -- "$TEMP" while true ; do case "$1" in -a|--a-long) echo "Option a" ; shift ;; -b|--b-long) echo "Option b, argument `$2'" ; shift 2 ;; -c|--c-long) # c has an optional argument. As we are in quoted mode, # an empty parameter will be generated if its optional # argument is not found. case "$2" in "") echo "Option c, no argument"; shift 2 ;; *) echo "Option c, argument `$2'" ; shift 2 ;; esac ;; --) shift ; break ;; *) echo "Internal error!" ; exit 1 ;; esac done echo "Remaining arguments:" for arg do echo '--> '"`$arg'" ; done
First of all what does the
shift program in the following line:
-a|--a-long) echo "Option a" ; shift ;;
Afterwards what is the purpose to use the
eval command in the following line:
eval set -- "$TEMP"
I tried to comment the line in script mentioned above and I got the following response:
$ ./getOptExample2.sh -a 10 -b 20 --a-long 40 -charem --c-long=echi Param: -a Option a Param: 10 Internal error!
But if I uncomment it it runs like a charm:
Option a Option b, argument `20' Option a Option c, argument `harem' Option c, argument `echi' Remaining arguments: --> `10' --> `40'
One of the many things that
getopt does while parsing options is to rearrange the arguments, so that non-option arguments come last, and combined short options are split up. From
Output is generated for each element described in the previous section. Output is done in the same order as the elements are specified in the input, except for non-option parameters. Output can be done in compatible (unquoted) mode, or in such way that whitespace and other special characters within arguments and non-option parameters are preserved (see QUOTING). When the output is processed in the shell script, it will seem to be composed of distinct elements that can be processed one by one (by using the shift command in most shell languages). [...] Normally, no non-option parameters output is generated until all options and their arguments have been generated. Then '--' is generated as a single parameter, and after it the non-option parameters in the order they were found, each as a separate parameter.
This effect is reflected in your code, where the option-handling loop assumes that all option arguments (including arguments to options) come first, and come separately, and are finally followed by non-option arguments.
TEMP contains the rearranged, quoted, split-up options, and using
eval set makes them script arguments.
eval? You need a way to safely convert the output of
getopt to arguments. That means safely handling special characters like spaces,
*, etc. To do that,
getopt escapes them in the output for interpretation by the shell. Without
eval, the only option is
set $TEMP, but you’re limited to what’s possible by field splitting and globbing instead of the full parsing ability of the shell.
Say you have two arguments. There is no way to get those two as separate words using just field splitting without additionally restricting the characters usable in arguments (e.g., say you set IFS to
:, then you cannot have
: in the arguments). So, you need to able to escape such characters and have the shell interpret that escaping, which is why
eval is needed. Barring a major bug in
getopt, it should be safe.
shift, it does what it always does: remove the first argument, and shift all arguments (so that what was
$2 will now be
$1). This eliminates the arguments that have been processed, so that, after this loop, only non-option arguments are left and you can conveniently use
$@ without worrying about options.
The script works correctly when it gives you an error for
-a 10. The
-a option needs no parameter in this script. You should only use
The shift described in the man page as the following:
shift [n] The positional parameters from n+1 ... are renamed to $1 .... Parameters represented by the numbers $# down to $#-n+1 are unset. n must be a non-negative number less than or equal to $#. If n is 0, no parameters are changed. If n is not given, it is assumed to be 1. If n is greater than $#, the positional parameters are not changed. The return status is greater than zero if n is greater than $# or less than zero; otherwise 0.
So basically it drops -a and shift the remaining arguments so the second parameter will be $1 in the next cycle.
-- is also described in the man page:
-- A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments. An argument of - is equivalent to --.
Afterwards what is the purpose to use the eval command in the following line:
eval set -- "$TEMP"
The util-linux version of
getopt produces output that’s usable as input to the shell. It surrounds strings containing whitespace with quotes, and handles escaping of literal quotes and other special characters.
$ getopt -o a:b -- -a 'foo bar' -b "single'quotes'here" -a 'foo bar' -b -- 'single'''quotes'''here'
The quotes won’t be processed from the result of a plain expansion, but a full round of parsing is required. And that’s what
If the output is assigned to
$tmp, then after
eval set -- "$tmp", the positional parameters
$2, … contain
single'quotes'here, and that’s relatively easy to process in a loop.
set -- $tmp would set the positional parameters to
bar', etc…, which is not what you want. (Also you’d get globbing on top, if one of the args was e.g.
The problems are similar to those in How can we run a command stored in a variable?, both cases involve lists of arbitrary strings.
Note that the behaviour where
getopt produces shell-quoted output is specific to the util-linux version of it. Other systems often have a
getopt that is used without
eval, and which happily breaks arguments that contain whitespace or look like globs. As in this one, there’s no way to tell from the output that
foo bar is supposed to be a single argument.
$ getopt a:b -a 'foo bar' -b "single'quotes'here" -a foo bar -b -- single'quotes'here
getopt, make sure to use the
--test option first to see if you have the safe version that can deal with arbitrary strings.