Looping through variables which is an output of another command

Hello I am learning Scripting here. I am trying to write a simple script using the ‘for’ loop.

I have hundreds of folders in a folder called user.
if i run this command i get a list of folders that i need to move to another folder

cat failed | awk -F ":" '/FAILED/ {print $1}' |  uniq

i.e folders under users that have failed a task have to be moved to users/failed/failedusers

what i am currently doing is i am creating a new file first using

cat failed | awk -F ":" '/FAILED/ {print $1}' |  uniq > failedusers

and then i move the folders using the following command

while read line; do cp -r users/$line users/failed; done < failedusers

My question here is can i do the same using just the for command instead of writing the output to another file and using the while read command to get it done?

for example somehow assign a value to a variable in a loop like

faileduser=`cat failed | awk -F ":" '/FAILED/ {print $1}' |  uniq`

and then write something like

mv users/$faileduser user/failed/$faileduser

i am getting all kinds of errors when i am trying to write something like above.


Asked By: ranjit abraham


With GNU xargs and a shell with support for ksh-style process substitution, you can do:

xargs -rd 'n' -I USER -a <(awk -F : '/FAILED/ {print $1}' failed | sort -u
  ) cp -r users/USER user/failed/USER

With zsh, you could do:

faileduser=( ${(f)"$(awk -F : '/FAILED/ {print $1}' failed | sort -u)"} )
autoload zargs
zargs -rI USER -- $faileduser -- cp -r users/USER user/failed/USER

Assuming you want to copy USER to user/failed/USER, that is, copy it into user/failed, you could also do (still with zsh):

(( $#faileduser )) && cp -r users/$^faileduser user/failed/

With the bash shell, you could do something similar with:

readarray -t faileduser < <(awk -F : '/FAILED/ {print $1}' failed | sort -u)
(( ${#faileduser[@]} )) &&
  cp -r "${faileduser[@]/#/user/}" user/failed/

Or get awk to prepend the user/ to all the user names:

readarray -t faileduser < <(awk -F : '/FAILED/ {print "user/"$1}' failed | sort -u)
(( ${#faileduser[@]} )) &&
  cp -r "${faileduser[@]}" user/failed/

With a for loop, with Korn-like shells (including bash, and would also work with zsh) the syntax would be:

for user in "${faileduser[@]}"; do
  cp -r "user/$user" "user/failed/$user"

Which in zsh could be shortened to:

for user ($faileduser) cp -r user/$user user/failed/$user


faileduser=`cat failed | awk -F ":" '/FAILED/ {print $1}' |  uniq`

(`...` being the archaic and deprecated form of command substitution. Use $(...) instead).

You’re storing awk‘s output in a scalar, not array variable.

In zsh, you can split it on newline with the f parameter expansion flag like we do directly above on the command substitution:

array=( ${(f)faileduser} )

In bash (or ksh), you could use the split+glob operator after having disabled glob and tuned split:

set -o noglob
array=( $faileduser )

(yes, in bash, leaving a parameter expansion unquoted invokes an implicit split+glob operator (!), a misfeature inherited from the Bourne shell, fixed in zsh and most modern non-Bourne-like shells).

Answered By: Stéphane Chazelas
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.