Execute copy commands from file

I have a DE_CopyOldToNew.txt file with a whole bunch of copy commands for copying old file names to new file names.

The file contains rows like :

cp /migrationfiles/Company Name GmbH/2014.138_Old File Name.pdf /appl/data/docs/202403/DE_2014.138_NewFile_1.pdf
cp /migrationfiles/Company Name GmbH/2014.139_Old File Name.pdf /appl/data/docs/202403/DE_2014.139_NewFile_1.pdf

In my shell script I am iterating over each row and execute it.

echo "Start!"
while read -r line
  command "$line"
done < /tmp/DE_CopyOldToNew.txt

When executing the script I am getting the following for each row that was read…

 : No such file or directory6: cp /migrationfiles/Company Name GmbH/2014.138_Old File Name.pdf /appl/data/docs/202403/DE_2014.138_NewFile_1.pdf

When executing the rows manually from the prompt it works without errors…

Asked By: Rico Strydom


The problem is that you are reading the entire line into one shell variable line, which you then use quoted (i.e. as "$line").

While it is usually the right thing to double-quote shell variables when using them, and in particular when dealing (as you do) with filenames that contain spaces or other "problematic" characters, in this case it means that the entire line consisting of the cp command and its arguments is treated as one single shell token in your command statement.

This means that in the loop, you are not executing the cp command with arguments /migrationfiles/Company Name GmbH/2014.138_Old File Name.pdf and /appl/data/docs/202403/DE_2014.138_NewFile_1.pdf, but the shell tries to run an executable named

cp /migrationfiles/Company Name GmbH/2014.138_Old File Name.pdf /appl/data/docs/202403/DE_2014.138_NewFile_1.pdf

where the spaces that are meant to separate command from arguments are interpreted as being part of the command name. This executable of course doesn’t exist.

What you can instead do is read the lines into three variables, as in:

echo "Start!"
while read cmd arg1 arg2
  command "$cmd" "$arg1" "$arg2"
done < /tmp/DE_CopyOldToNew.txt

assuming that the structure is always of the form as indicated in your example.

Note that this read command does not contain the -r option, because that would inhibit interpreting the as escape character for the following space, which however we need the shell to do here.

Answered By: AdminBee

I wouldn’t let the TXT file do any action, but treat it as a source of data, so use the script not only to execute what needs to be executed but also as a control of the data:

# Read each line of the file
while IFS= read -r line; do
    # Get the files using awk with ' /' as separator.
    # $line could have escaped spaces with ' ' so we use parameter substitution →
    # → to substitute them to unescaped spaces: "${line//\ / }".
    source_file=$(awk -F' /' '{ print "/"$2 }' <<<"${line//\ / }") # source file
    destination=$(awk -F' /' '{ print "/"$3 }' <<<"$line") # destination
    # For example you can control that the source file exists
    if [[ ! -f "$source_file" ]]; then
        echo "The file $source_file does not exists!"
    # The cp command is executed here
    cp "$source_file" "$destination"
done </tmp/DE_CopyOldToNew.txt

You say,

I have a DE_CopyOldToNew.txt file with a whole bunch of copy commands for copying old file names to new file names.

… and you say you want to execute these cp commands.

You are in luck because you have a shell script, i.e., a file containing a set of commands to be executed in order by a shell. All you need to do is to let a shell execute the commands in the script:

sh DE_CopyOldToNew.txt

This would start sh, a basic shell, and have it execute the file’s contents, command by command.

Answered By: Kusalananda