How to count separate line entries inside a pipe

Use shell script to count separate line entries inside a pipe to see how many entries got filtered out. It can be useful in some scenarios, especially for testing.

Shell script

The complete shell script.

#!/bin/bash
# Read standard input in a pipeline, count lines/characters and pass to the next command

# variables
prefix=""
n_lines=0
n_chars=0

# define usage function
function usage {
  echo "Read standard input in a pipeline, count lines/characters and pass to the next command"
  echo "Use -p parameter to provide a prefix"
  echo "Results will be shown on standard error"
}

# parse arguments
while getopts "p:" option; do
  case $option in
    "p")
      prefix=${OPTARG}
      ;;
    esac
done

if [ "$#" -eq "0" ] || [ "$#" -eq "2" ]; then
  if [ ! -t 0 ] && [ -p /dev/stdin ]; then
    file_read=false
    until $file_read; do
      read -r -u 0 line || file_read=true
      if [  "$file_read" == false ]; then
        n_lines=$(expr $n_lines \+ 1)
        l_chars=$(printf "%s\n" "$line" | wc -c)
        n_chars=$(expr $n_chars \+ $l_chars)
        printf "%s\n" "$line" > /dev/stdout
      elif ([ "$file_read" == true ] && [ -n "$line" ]); then
        n_lines=$(expr $n_lines \+ 1)
        l_chars=$(printf "%s" "$line" | wc -c)
        n_chars=$(expr $n_chars \+ $l_chars)
        printf "%s\n" "$line" > /dev/stdout
      fi
    done
  else
    usage
    exit
  fi
fi

# set single or plural words
if [ "$n_lines" -eq "1" ]; then
  line_single_or_plural="line"
else
  line_single_or_plural="lines"
fi

if [ "$n_chars" -eq "1" ]; then
  char_single_or_plural="character"
else
  char_single_or_plural="characters"
fi

# display lines/characters count
if [ -n "$prefix" ]; then
  echo -n "${prefix}: " > /dev/stderr
fi
echo "Counted $n_lines $line_single_or_plural and $n_chars $char_single_or_plural" > /dev/stderr

Examples

Usage information.

$ pipe_count.sh 
Read standard input in a pipeline, count lines/characters and pass to the next command
Use -p parameter to provide a prefix
Results will be shown on standard error

Sample standalone usage inside a pipeline with df command.

$ df -h | pipe_count.sh -p "_df" | sed 1d | pipe_count.sh -p "_df w/o header" | awk '$5 > 10 { print }' | pipe_count.sh -p "_df filtered for usage above 10%"
_df: Counted 11 lines and 412 characters
_df w/o header: Counted 10 lines and 361 characters
tmpfs                              788M   79M  709M  11% /run
/dev/mapper/vg-root                225G   32G  183G  15% /
tmpfs                              3,9G  273M  3,6G   7% /dev/shm
/dev/sda2                          465M  363M   74M  84% /boot
_df filtered for usage above 10%: Counted 4 lines and 150 characters

Sample usage inside a pipeline with ss command.

$ ss -t -n -H | pipe_count.sh -p "_ss tcp" | grep -v 127.0.0.1 | pipe_count.sh -p "_ss w/o localhost" | grep 443 | pipe_count.sh -p "_ss filtered for port 443"
_ss tcp: Counted 27 lines and 1211 characters
_ss w/o localhost: Counted 13 lines and 623 characters
ESTAB      0      0      192.168.1.177:59754              216.58.209.46:443
ESTAB      0      0      192.168.1.177:56682              46.165.244.206:443
_ss filtered for port 443: Counted 2 lines and 97 characters

Additional notes

I presume that /dev/stderr exists. I do not check if it is available, but maybe I should verify this specific situation.

About Milosz Galazka

Milosz is a Linux Foundation Certified Engineer working for a successful Polish company as a system administrator and a long time supporter of Free Software Foundation and Debian operating system.