What is the effect of 'exec' shell built-in in a background pipeline?

Through iterative process I’ve created a following script that works exactly as intended – terminates VPN when the main process is interrupted and passes a password to stdin of openconnect.

#!/bin/sh
cat password | exec openconnect --user=example vpn.example.com &
trap "kill $!" EXIT
wait

I’ve accidentally failed to remove exec built-in from the pipeline. I had an understanding that exec replaces a running shell process with a given command, so I would expect trap and wait built-ins to never be executed, and yet, they do.
Which process am I accidentally replacing and what are consequences of removing the exec keyword?
Should I use exec in my pipelines to avoid spawning of additional shell processes?

Bash tag is due to:

% sh --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin21)
Copyright (C) 2007 Free Software Foundation, Inc.
Asked By: Basilevs

||

exec is a bit of a misnomer for that builtin. In fact, it should rather have been called nofork.

When interpreting the

cmd

shell code, where cmd is not a shell builtin (and there is no &),
the shell fork()s into two processes: parent and child.

  • in the child process,
    it calls execle("/path/to/cmd", ["cmd", 0], environ).
  • in the parent, it wait*()s for the child to terminate.

In:

exec cmd

The execle() is done in the shell process. 
There’s no fork, the process that used to run code from the shell interpreter now runs code for cmd, and the shell is gone.

When cmd is a builtin or function, the behaviour depends on the shell;
it either

  • emulates the exec by running the builtin and then exiting, or
  • doesn’t run the builtin but runs a cmd executable by the same name if it exists or exits with a "command not found" error if not.

In:

cmd &

It’s the same as in cmd except the parent doesn’t wait*() for the child. There are a few side differences, such as:

  • if the shell is interactive, the cmd‘s process group is not made the foreground process group of the terminal (so it’s not killed upon Ctrl+C or suspended upon Ctrl+Z for instance, but is suspended if tries to read from the terminal).
  • in non-interactive shells, with most shells,
    in the child process that will execute cmd,
    stdin is redirected to /dev/null (unless otherwise redirected),
    and the SIGINT and SIGQUIT signals are ignored.

In:

exec cmd &

The exec is mostly superfluous. & means there will be a child process that is not waited for, so the exec (more like nofork) is pointless as there’s already been a fork().

The only point of doing that would be if you want to bypass builtins/functions in shells where exec does it.

In:

cmd1 | cmd2

cmd1 and cmd2 run concurrently, so in different processes.

Whether cmd2 runs in a subshell or not (which matters when it’s a builtin, function or compound command) depends on the shell. In practice, in all shells, cmd1 runs in a subshell. 
In any case, if cmd1 or cmd2 is an external command, a child process will be need to be forked to run it. Whether the parent waits for cmd1 and cmd2 or just cmd2 depends on the shell.

cmd1 | exec cmd2
echo done

Whether exec cmd2 terminates the shell or not depends on whether that’s run in a subshell or not, as per above. In AT&T implementations of ksh, zsh or bash -O lastpipe (when non-interactive), that will (and you’ll also notice that

zsh -c 'printf "%sn" foo | IFS= read -r var; printf "%sn" "$var"'

outputs foo, as read was not run in a child process;
same for the other two); in others, generally not.

In cmd1 | cmd2 &, the whole pipeline is run in the background and processes in it are not waited for. Like in cmd &, there will have to be a fork(), so exec makes no difference other than the side effects when the commands are builtins or functions as noted above.

So in:

#!/bin/sh
cat password | exec openconnect --user=example vpn.example.com &
trap "kill $!" EXIT
wait

Unless openconnect happens to be a builtin of your shell, or sh is actually bash and there’s an exported openconnect bash function in the environment, the exec will make no difference.

Now, not sure why you’re trying to concatenate a single file, but note that $! will be the pid of the process that runs openconnect. So, in kill $!, cat won’t be killed (not really a problem as I assume the password file is short enough to fit in a pipe buffer so cat will probably have finished its job very soon after starting, and even if not would be killed with a SIGPIPE when trying to write to the pipe after the process running openconnect is gone).

Answered By: Stéphane Chazelas
Categories: Answers Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.