Categories
SysOps

How to inspect a pipeline

A pipeline is a sequence of multiple commands separated by the control operators. It is easy to understand as long as you see the whole picture, but I will show you how to inspect it when you have a single process ID.

Initial notes

At first, see how to inspect default file descriptors.

$ ls -l /dev/std*
lrwxrwxrwx 1 root   root   15 cze 29 20:51 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root   root   15 cze 29 20:51 /dev/stdin  -> /proc/self/fd/0
lrwxrwxrwx 1 root   root   15 cze 29 20:51 /dev/stdout -> /proc/self/fd/1
$ ls -l /proc/self/fd/*
lrwx------ 1 milosz milosz 64 lip  1 23:14 /proc/self/fd/0 -> /dev/pts/9
lrwx------ 1 milosz milosz 64 lip  1 23:14 /proc/self/fd/1 -> /dev/pts/9
lrwx------ 1 milosz milosz 64 lip  1 23:14 /proc/self/fd/2 -> /dev/pts/9

The procedure

Pipeline to inspect.

$ find / -newerct "2019-01-01 00:00" 2>/dev/null | tee files.log

find process ID is 22699. You can see that standard output is redirected to a pipe 23367059 and standard error to /dev/null. You can ignore other file descriptors as these are related to find operations.

$ tr '\0' ' ' </proc/26999/cmdline
find / -newerct 2019-01-01 00:00
$ ls -l /proc/22699/fd/*
lrwx------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/0 -> /dev/pts/10
l-wx------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/1 -> 'pipe:[23367059]'
lrwx------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/2 -> /dev/null
lr-x------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/3 -> /home/milosz/
lr-x------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/4 -> /proc/3862/task/3916/net
lr-x------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/6 -> /proc/3862/task/3916
lr-x------ 1 milosz milosz 64 lip  1 23:55 /proc/22699/fd/8 -> /proc/3862/task

Inspect pipe 23367059.

$ lsof -d0-1024 -w | awk '$NF == "pipe" && $8 == "23367059" {print}'
find      22699 milosz    1w     FIFO               0,12       0t0   23367059 pipe
tee       22700 milosz    0r     FIFO               0,12       0t0   23367059 pipe

find process is writing to a pipe using the file descriptor 1 and tee is reading from it using the file descriptor 0.

Confirm your findings by inspecting tee process (PID 22700). The command is reading from the pipe 23367059, writing to a terminal and files.log file.

$ tr '\0' ' ' </proc/22700/cmdline
tee files.log
$ ls -l /proc/22700/fd/*
lr-x------ 1 milosz milosz 64 lip  1 23:55 /proc/22700/fd/0 -> 'pipe:[23367059]'
lrwx------ 1 milosz milosz 64 lip  1 23:55 /proc/22700/fd/1 -> /dev/pts/10
lrwx------ 1 milosz milosz 64 lip  1 23:55 /proc/22700/fd/2 -> /dev/pts/10
l-wx------ 1 milosz milosz 64 lip  1 23:55 /proc/22700/fd/3 -> /home/milosz/files.log

I could write more examples, but this procedure is the same for every other case.

Additional information

You can display pipe endpoints to get better overview.

$ lsof -w +E -d0-1024 | grep 23367059
find      22699 milosz    1w     FIFO               0,12       0t0   23367059 pipe 22700,tee,0r
tee       22700 milosz    0r     FIFO               0,12       0t0   23367059 pipe 22699,find,1w