Bash – How to pipe AWK command output to Python as first argument


I have a plain text file input.txt which looks like this:

D000001 D000001 44 1975
D000001 D000408 1 1983
D000001 D000641 1 1977
D000001 D000900 27 1975

I process this file using this simple AWK line:

awk '{if ($4 == 1975) print $1,$2,$3}' input.txt

Then I have a Python script which accept a file as the first command line argument:

#!/usr/bin/env python3

import sys

file_name = sys.argv[1]

I wonder it is possible to pipe AWK output to Python program as file argument and how to do that?

Best Answer

If you want to use a pipe, then your python script would have to read from stdin. Your script doesn't do that. Instead it expects a file name on the command line. This can be accomplished using a shell feature called process substitution to connect the two together: <(awk '{if ($4 == 1975) print $1,$2,$3}' input.txt)

<(...) denotes process substitution. What happens here is that the shell creates a file-like object that contains the output of the awk command. This file-like object even has a name. If you run the script, the output will see its name, passed to python as sys.argv[1], is something like: