Combine two files as part of a command line parameter
I am trying to use csvgrep (part of csvkit) to extract rows matching one of two files. Unfortunately, there is no command line option that allows two matching files (like grep does). Only,
csvgrep -f list1.txt -c 2 myfile.csv
Is there a way to concatenate list1.txt with list2.txt as part of the command?
csvgrep -f (list1.txt+list2.txt) -c 2 myfile.csv
The only way I can think of doing this is to use a temporary intermediate file, but I would rather avoid that if possible.
Assuming you want the command to see a "filename" that gives the contents of both files when read, then process substitution and
cat are what you want:
csvgrep -f <(cat list1.txt list2.txt) -c 2 myfile.csv
Basically, process substitution is a generalized pipeline, it runs the commands inside, and arranges for their output to be available via magic file that can be opened like any regular file. The expansion itself expands to the name of that magic file. (i.e.
echo <(echo foo) prints something like
cat <(echo foo) prints
Many programs allow
- to be used as a filename to represent standard input or output (depending on whether it’s an input or output file). So you can try piping to
csvgrep and using
cat list1.txt list2.txt | csvgrep -f - -c 2 myfile.csv
If it works, it has an advantage over the process substitition method because it doesn’t depend on a shell-specific extension.
If that doesn’t work, you could use
/dev/stdin. This is a standard Linux device that automatically refers to the process’s standard input.
Note that all these solutions depend on
csvgrep not needing to seek in the
-f file, because the output of
cat is being sent through a pipe. But this seems like a safe assumption.