Is there any way to read lines from command output?
I have a pre process command to output a file
./preprocess.sh > preprocessed_file
and the preprocessed_file will be used like this
while read line
do
./research.sh $line &
done < preprocessed_file
rm -f preprocessed_file
Is there any way to direct the output to the while read line part instead of outputting to the preprocessed_file? I think there should be a better way other than using this temp preprocessed_file.
You can use bash process substitution:
while IFS= read -r line; do
./research.sh "$line" &
done < <(./preprocess.sh)
Some advantages of process substitution:
- No need to save temporary files.
- Better performance. Reading from another process often faster than writing to disk, then read back in.
- Save time to computation since when it is performed simultaneously with parameter and variable expansion, command substitution, and arithmetic expansion
Yes! You can use a process pipe |.
./preprocess.sh |
while IFS= read -r line
do
./research.sh "$line" &
done
A process pipe passes the standard output (stdout) of one process to the standard input (stdin) of the next.
You can optionally put a newline character following a | and extend the command to the next line.
Note: a|b is equivalent to b < <(a), but without the magic files, and in a more readable order, especially when the pipeline gets longer.
a|b|c is equivalent to c < <(b < <(a))
and
a|b|c|d|e is e < < (d < <(c < <(b < <(a))))