Double backslash disappears when printed in a loop

I have a script that joins together various lists of data fields which then needs to have a few more columns added. The file generated looks like this:

$ cat compiled.csv
"name":"Network Rule 1", "description":"Network Rule 1", "from":["internal"], "source":["any"], "user":["domain\network_user1"], "to":["external"], "destination":["host.example.com","10.1.2.1"], "port":["8443","22"],
"name":"Network Rule 2", "description":"Network Rule 2", "from":["internal"], "source":["any"], "user":["domain\network_user2"], "to":["external"], "destination":["host.example.com","10.2.1.1"], "port":["23","25"],
"name":"Network Rule 3", "description":"Network Rule 3", "from":["internal"], "source":["any"], "user":["domain\network_user3"], "to":["external"], "destination":["host.example.com","10.3.4.1"], "port":["80","143"],

I’m trying to append a few more fields (all the same) to the list; these fields would be something like…

"access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"

The final output should look like this:

"name":"Network Rule 1", "description":"Network Rule 1", "from":["internal"], "source":["any"], "user":["domain\network_user1"], "to":["external"], "destination":["host.example.com","10.1.2.1"], "port":["8443","22"], "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"
"name":"Network Rule 2", "description":"Network Rule 2", "from":["internal"], "source":["any"], "user":["domain\network_user2"], "to":["external"], "destination":["host.example.com","10.2.1.1"], "port":["23","25"], "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"
"name":"Network Rule 3", "description":"Network Rule 3", "from":["internal"], "source":["any"], "user":["domain\network_user3"], "to":["external"], "destination":["host.example.com","10.3.4.1"], "port":["80","143"], "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"

When I try to append the fields in a loop, my double backslash disappears both when running the following command in a script and directly in the shell.

while read LINE; do
 echo $LINE "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"
done < compiled.csv > completed_list.csv

Instead, this results in the following example where the username double backslash has disappeared.

"name":"Network Rule 1", "description":"Network Rule 1", "from":["internal"], "source":["any"], "user":["domainnetwork_user1"], "to":["external"], "destination":["host.example.com","10.1.2.1"], "port":["8443","22"], "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"

I’m guessing something is wrong with using echo to print the whole line, but what is a way to work around that?

Thank you in advance.

Asked By: vxla

||

read by default interprets backslashes that can be used to escape IFS characters from delimiting, the -r option turns that off.

$ read a b  <<< '1 2 3'
$ printf "<%s>n" "$a"
<1 2>
$ read -r a b  <<< '1 2 3'
$ printf "<%s>n" "$a"
<1>

Also, even with just one variable name given, read will remove leading and trailing whitespace (if present in IFS), so you may want to explicitly set IFS to the empty string for it to get an unmodified input line.

Depending on your shell and the settings, echo can also process backslashes when printing. Better use printf "%sn" "..." instead.

Also, if you print double quotes, it may be easier to single-quote the whole string, instead of escaping each one separately.

So, maybe:

while IFS= read -r LINE; do
    echo "$LINE"' "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"'
done < compiled.csv > completed_list.csv

Then again, since it looks like you’re just adding a fixed string to the end of each line, you could use sed instead. It’ll likely be faster then the shell:

sed -e 's/$/ "access":"allow", "time":"00:00:00-23:59:59", "notify":"yes"/' < compiled.csv > completed_list.csv
Answered By: ilkkachu
Categories: Answers Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.