How to log dropped connections from iptables firewall using netfilter userspace logging daemon

Log dropped connections from iptables firewall using netfilter userspace logging daemon for further analysis and troubleshooting.

Check out if nf_log modules for IPv4 and IPv6 are loaded.

$ cat /proc/net/netfilter/nf_log 
 0 NONE ()
 1 NONE ()
 2 NONE ()
 3 NONE ()
 4 NONE ()
 5 NONE ()
 6 NONE ()
 7 NONE ()
 8 NONE ()
 9 NONE ()
10 NONE ()
11 NONE ()
12 NONE ()

Load kernel nf_log modules for IPv4 and IPv6.

$ sudo modprobe nf_log_ipv6
$ sudo modprobe nf_log_ipv4

Verify that modules are loaded.

$ cat /proc/net/netfilter/nf_log 
 0 NONE ()
 1 NONE ()
 2 nf_log_ipv4 (nf_log_ipv4)
 3 NONE ()
 4 NONE ()
 5 NONE ()
 6 NONE ()
 7 NONE ()
 8 NONE ()
 9 NONE ()
10 nf_log_ipv6 (nf_log_ipv6)
11 NONE ()
12 NONE ()

Ensure that these modules will load at boot.

$ cat << EOF | sudo tee /etc/modules-load.d/nf_log.conf
nf_log_ipv4
nf_log_ipv6
EOF

Install netfilter userspace logging daemon.

$ sudo apt-get install ulogd2

Define two new plugin stacks inside /etc/ulogd.conf file.

stack=firewall11:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu11:LOGEMU
stack=firewall12:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,print1:PRINTPKT,emu12:LOGEMU

Define input plugins using specified netlink group inside /etc/ulogd.conf file.

[firewall11]
group=11

[firewall12]
group=12

Define output plugins inside /etc/ulogd.conf file.

[emu11]
file="/var/log/ulog/firewall.log"
sync=1

[emu12]
file="/var/log/ulog/firewall-ssh-drop.log"
sync=1

Restart service.

$ sudo systemctl restart ulogd2

Create iptables firewall that will allow already established connections, incoming icmp and ssh, outgoing icmp, ntp, dns, ssh, http and https. It will also log invalid packets and those dropped ones.

# Flush rules and delete custom chains
iptables -F
iptables -X

# Define chain to allow particular source addresses
iptables -N chain-incoming-ssh 
iptables -A chain-incoming-ssh -s 192.168.1.149 -j ACCEPT -m comment --comment "local access"
iptables -A chain-incoming-ssh -j NFLOG --nflog-prefix "[fw-inc-ssh]:" --nflog-group 12
iptables -A chain-incoming-ssh -j DROP

# Define chain to log and drop incoming packets
iptables -N chain-incoming-log-and-drop
iptables -A chain-incoming-log-and-drop -j NFLOG --nflog-prefix "[fw-inc-drop]:" --nflog-group 11
iptables -A chain-incoming-log-and-drop -j DROP

# Define chain to log and drop outgoing packets
iptables -N chain-outgoing-log-and-drop
iptables -A chain-outgoing-log-and-drop -j NFLOG --nflog-prefix "[fw-out-drop]:" --nflog-group 11
iptables -A chain-outgoing-log-and-drop -j DROP 

# Drop invalid packets
iptables -A INPUT -m conntrack --ctstate INVALID -j chain-incoming-log-and-drop

# Accept everthing on loopback
iptables -A INPUT  -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# ACCEPT incoming packets for established connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Accept incoming ICMP
iptables -A INPUT -p icmp -j ACCEPT

# Accept incoming SSH
iptables -A INPUT -p tcp --dport 22 -j chain-incoming-ssh

# Accept outgoing packets for established connections
iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Accept outgoing DNS
iptables -A OUTPUT -p tcp --dport 53 -j ACCEPT
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT

# Accept outgoing NTP
iptables -A OUTPUT -p tcp --dport 123 -j ACCEPT
iptables -A OUTPUT -p udp --dport 123 -j ACCEPT

# Accept outgoing HTTP/S
iptables -A OUTPUT -p tcp --dport 80  -j ACCEPT
iptables -A OUTPUT -p tcp --dport 443 -j ACCEPT

# Accept outgoing SSH
iptables -A OUTPUT -p tcp --dport 22  -j ACCEPT

# Accept outgoing ICMP
iptables -A OUTPUT -p icmp -j ACCEPT

# Log not accounted outgoing traffic
iptables -A OUTPUT -j chain-outgoing-log-and-drop

# Log not accounted forwarding traffic
iptables -A FORWARD -j chain-incoming-log-and-drop

# Drop everything else
iptables -P INPUT   DROP
iptables -P FORWARD DROP
iptables -P OUTPUT  DROP

List all firewall rules to verify that executed commands are applied as desired.

$ sudo iptables -L -v -n
Chain INPUT (policy DROP 136 packets, 7799 bytes)
 pkts bytes target                       prot opt in     out     source               destination
    0     0 chain-incoming-log-and-drop  all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
    0     0 ACCEPT                       all  --  lo     *       0.0.0.0/0            0.0.0.0/0
 3922 1478K ACCEPT                       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT                       icmp --  *      *       0.0.0.0/0            0.0.0.0/0
   14   896 chain-incoming-ssh           tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target                       prot opt in     out     source               destination
    0     0 chain-incoming-log-and-drop  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target                       prot opt in     out     source               destination
    0     0 ACCEPT                       all  --  *      lo      0.0.0.0/0            0.0.0.0/0
 2811  370K ACCEPT                       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 ACCEPT                       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
   18  1106 ACCEPT                       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT                       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:123
    1    76 ACCEPT                       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:123
    3   180 ACCEPT                       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80
    1    60 ACCEPT                       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443
    0     0 ACCEPT                       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    0     0 ACCEPT                       icmp --  *      *       0.0.0.0/0            0.0.0.0/0
   10   936 chain-outgoing-log-and-drop  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain chain-incoming-log-and-drop (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 NFLOG      all  --  *      *       0.0.0.0/0            0.0.0.0/0            nflog-prefix  "[fw-inc-drop]:" nflog-group 11
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain chain-incoming-ssh (1 references)
 pkts bytes target     prot opt in     out     source               destination
    2   176 ACCEPT     all  --  *      *       192.168.1.149        0.0.0.0/0            /* local access */
   12   720 NFLOG      all  --  *      *       0.0.0.0/0            0.0.0.0/0            nflog-prefix  "[fw-inc-ssh]:" nflog-group 12
   12   720 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain chain-outgoing-log-and-drop (1 references)
 pkts bytes target     prot opt in     out     source               destination
   10   936 NFLOG      all  --  *      *       0.0.0.0/0            0.0.0.0/0            nflog-prefix  "[fw-out-drop]:" nflog-group 11
   10   936 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Sample log entries.

$ sudo tail /var/log/firewall.log 
Dec 31 16:30:26 debian [fw-out-drop]: IN= OUT=eth0 MAC= SRC=192.168.1.131 DST=84.16.240.28 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=30946 DF PROTO=TCP SPT=44172 DPT=8080 SEQ=2283062702 ACK=0 WINDOW=29200 SYN URGP=0 UID=0 GID=0 MARK=0 
Dec 31 16:30:26 debian [fw-out-drop]: IN= OUT=eth0 MAC= SRC=192.168.1.131 DST=84.16.240.28 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=30947 DF PROTO=TCP SPT=44172 DPT=8080 SEQ=2283062702 ACK=0 WINDOW=29200 SYN URGP=0 UID=0 GID=0 MARK=0 
Dec 31 16:30:29 debian [fw-out-drop]: IN= OUT=eth0 MAC= SRC=192.168.1.131 DST=84.16.240.28 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=30948 DF PROTO=TCP SPT=44172 DPT=8080 SEQ=2283062702 ACK=0 WINDOW=29200 SYN URGP=0 UID=0 GID=0 MARK=0 
Dec 31 16:31:44 debian [fw-out-drop]: IN= OUT=eth0 MAC= SRC=192.168.1.131 DST=84.16.240.28 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=63923 DF PROTO=TCP SPT=53568 DPT=8081 SEQ=4276941016 ACK=0 WINDOW=29200 SYN URGP=0 UID=0 GID=0 MARK=0 
Dec 31 16:31:44 debian [fw-out-drop]: IN= OUT=eth0 MAC= SRC=192.168.1.131 DST=84.16.240.28 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=63924 DF PROTO=TCP SPT=53568 DPT=8081 SEQ=4276941016 ACK=0 WINDOW=29200 SYN URGP=0 UID=0 GID=0 MARK=0 
$ sudo tail /var/log/ulog/firewall-ssh-drop.log
Dec 31 16:30:37 debian [fw-inc-ssh]: IN=eth0 OUT= MAC=52:54:00:26:97:1f:b8:27:eb:0e:4e:b6:08:00 SRC=192.168.1.252 DST=192.168.1.131 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=61074 DF PROTO=TCP SPT=55360 DPT=22 SEQ=1657227273 ACK=0 WINDOW=29200 SYN URGP=0 MARK=0 
Dec 31 16:30:38 debian [fw-inc-ssh]: IN=eth0 OUT= MAC=52:54:00:26:97:1f:b8:27:eb:0e:4e:b6:08:00 SRC=192.168.1.252 DST=192.168.1.131 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=61075 DF PROTO=TCP SPT=55360 DPT=22 SEQ=1657227273 ACK=0 WINDOW=29200 SYN URGP=0 MARK=0 
Dec 31 16:30:40 debian [fw-inc-ssh]: IN=eth0 OUT= MAC=52:54:00:26:97:1f:b8:27:eb:0e:4e:b6:08:00 SRC=192.168.1.252 DST=192.168.1.131 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=61076 DF PROTO=TCP SPT=55360 DPT=22 SEQ=1657227273 ACK=0 WINDOW=29200 SYN URGP=0 MARK=0 

Additional notes

Install tcpdump.

$ sudo apt-get install tcpdump

Use tcpdump to inspect particular netlink group.

$ sudo tcpdump -i nflog:12
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on nflog:12, link-type NFLOG (Linux netfilter log messages), capture size 262144 bytes
16:40:32.707438 IP raspberrypi.55362 > debian.local.ssh: Flags [S], seq 2507142205, win 29200, options [mss 1460,sackOK,TS val 105615136 ecr 0,nop,wscale 6], length 0
16:40:33.731425 IP raspberrypi.55362 > debian.local.ssh: Flags [S], seq 2507142205, win 29200, options [mss 1460,sackOK,TS val 105615240 ecr 0,nop,wscale 6], length 0
16:40:35.811425 IP raspberrypi.55362 > debian.local.ssh: Flags [S], seq 2507142205, win 29200, options [mss 1460,sackOK,TS val 105615448 ecr 0,nop,wscale 6], length 0
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

You will receive an error if netfilter userspace logging daemon or other application is already using it.

# tcpdump -i nflog:12
tcpdump: Can't listen on group group index: Operation not permitted

About Milosz Galazka

Milosz is a Linux Foundation Certified Engineer working for a successful Polish company as a system administrator and a long time supporter of Free Software Foundation and Debian operating system. He is also open for new opportunities and challenges.