pipe to a different file descriptor

Following up on Using while loop to ssh to multiple servers, i.e., for

while IFS= read -r -u9 HOST ; do ssh "$HOST" "uname -a" ; done 9< servers.txt

which reads from a different file descriptor (9),

How to make it read from a pipe of a different file descriptor?

If my pipe command is grep, what’s the easiest way for grep to write to the 9th file descriptor, and what’s the syntax of piping the 9th file descriptor?

Asked By: xpt

||

If you mean you want the while IFS= read -ru9 line; do ...; done to loop over the output of grep rather than the contents of servers.txt, then with bash, zsh or recent versions of ksh93¹, you can use redirection from a process substitution instead of a pipeline:

while IFS= read -ru9 host; do
  ssh "$host" 'some code for the remote shell'
done 9< <(grep something servers.txt)

grep will be run with its stdout redirected to the writing end of a pipe, and <(...) will be substituted with a file path which once opened (here on fd 9 using regular input redirection) will point to the reading end of that same pipe. Depending on the system and/or shell, that will be either a named pipe or /dev/fd/<a-number> or similar (such as /proc/self/fd/<a-number>).

Portably (in any POSIX-like shell), you can do:

{
  grep something servers.txt |
    while IFS= read<&9 -r host; do
      ssh "$host" 'code for the remote shell'
    done 9<&0 0<&3 3<&-
} 3<&0

That is using a regular pipeline, but doing some fd shuffling so the fd opened on the reading end of the pipe is moved to fd 9, and fd 0 is restored to the original stdin using a temporary fd 3.

Beware that in some shells, including bash (except when the lastpipe option is enabled and the shell is not interactive), the while loop will run in a subshell.

Instead of having read on fd 9, you could also simply

{
  grep something servers.txt |
    while IFS= read -r host; do
      ssh "$host" 'code for the remote shell' <&3 3<&-
    done
} 3<&0

Where we just restore ssh‘s stdin to the original one outside the pipeline using that fd 3 redirection trick again.

Instead of a while loop, on a GNU system, you can use xargs as:

xargs -rd 'n' -I HOST -a servers.txt ssh HOST '
  code for the remote shell
'

To get the list from servers.txt one per line and:

xargs -rd 'n' -I HOST -a <(grep something servers.txt) ssh HOST '
  code for the remote shell
'

To get it from the output of grep.

You can also add some -P0 to run all of them in parallel or -P4 to run up to 4 in parallel (but in either case all sharing the same stdin).

Another benefit is that if one of the ssh‘s fails, it will be reflected in the overall exit status.

Instead of using grep, you can also do shell pattern matching in the loop:

while IFS= read<&9 -r host; do
  case $host in
    (*something*) ssh "$host" 'some code for the remote shell';;
    (*else*) ssh "$host" 'some other code';;
  esac
done 9< servers.txt

(standard sh syntax and not running in a subshell).


¹ ksh is the shell that introduced process substitution in the mid-80s, but until relatively recently, you could not redirect fds from one there.

Answered By: Stéphane Chazelas
Categories: Answers Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.