Tar new volume script fails with "Syntax error: Bad fd number"
My shell script (see below) is failing partway through execution with "/bin/sh: 1: Syntax error: Bad fd number." This thread suggests that the problem is using the >& construction in sh, which in Ubuntu is linked to dash, which does not support >&. I am puzzled, because while the error message suggests that my script is running in sh, I am pretty confident that it is running in bash:
- The login shell from which I am executing it is bash, as indicated by echo $0.
- The first line of the script is #!/bin/bash
- If I insert readlink /proc/$$/exe into my script (as suggested here), the output is /usr/bin/bash.
- Both /bin/bash and /usr/bin/bash are actual executables, not links to other files.
So, if I’m wrong and it’s actually running in sh, why is it doing so? And if it is actually running in bash, why is it returning an error when that syntax is supposed to work in bash?
I’m running this in Ubuntu 22.04 using a 6.5.0-28 kernel.
Any feedback is appreciated.
The purpose of this script is to create a multipart tar (using this method) from a list of files and directories in a text file, capping the part size at 512 Gb. This should generate four files, but it consistently fails after the third part:
#!/bin/bash
tar -cvf /home/me/archive.tar --multi-volume --tape-length=512000 --new-volume-script='echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt
The full error text at failure is:
bin/sh: 1: Syntax error: Bad fd number
tar: ‘echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}’ command failed
tar: Error is not recoverable: exiting now
The format of file-list.txt is as follows:
/home/me/foo
/home/me/bar/file-1.xyz
/home/me/bar/file-2.xyz
The tar command uses /bin/sh to execute the command line given to --new-volume-script.
Evidence?
tar -cf /tmp/archive.tar --multi-volume --tape-length=1024 --new-volume-script='ps -f >/tmp/x' /etc 2>/dev/null
cat /tmp/x
Example output clearly showing that PID 25662 is executed with /bin/sh:
UID PID PPID C STIME TTY TIME CMD
chris 25210 25207 0 00:58 pts/0 00:00:00 -bash
chris 25651 25210 5 01:03 pts/0 00:00:00 tar -cf /tmp/archive.tar --multi-volume --tape-length=1024 --new-volume-script=ps -f >/tmp/x /etc
chris 25662 25651 0 01:03 pts/0 00:00:00 /bin/sh -c ps -f >/tmp/x
chris 25663 25662 99 01:03 pts/0 00:00:00 ps -f
Having established that tar invokes sh, and that sh is often implemented by dash rather than bash, we can move on to information about the file descriptor number:
tar -cf /home/me/archive.tar --multi-volume --tape=length=512000 --new-volume-script='echo "TAR_VOLUME=$TAR_VOLUME, TAR_FD=$TAR_FD." >&2; echo -n "/home/me/archive-${TAR_VOLUME}.tar" >&${TAR_FD}' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt
The resulting output is
TAR_VOLUME=2, TAR_FD=9.
TAR_VOLUME=3, TAR_FD=9.
TAR_VOLUME=4, TAR_FD=10.
And fails
TAR_VOLUME=5, TAR_FD=9.
TAR_VOLUME=6, TAR_FD=10.
And fails
Empirically we can suggest that a two digit file descriptor is causing the issue. Reading man dash leads us here:
The [n] is an optional number between
0and9, as in3(not[3]), that refers to a file descriptor
In other words a file descriptor for dash must be in the range 0 to 9, and file descriptors 10 and above cannot be referenced by number.
The solution here is to step outside dash into a shell with more flexibility, such as bash (I’ve split the command across several lines for improved readability):
tar -cvf /home/me/archive.tar --multi-volume --tape-length=512000 --new-volume-script='
exec bash -c "echo "/home/me/archive-$TAR_VOLUME.tar" >&$TAR_FD"
' --exclude-tag-all=donotcopy --exclude-backups -T /home/me/file-list.txt