Firehol ip based threat blocking

I’ve implemented blocking ips based on firehol’s ip lists. It works by checking every few minutes for changes to the list of block lists specified and creating ipsets that are checked before allowing connections in or out. It seems to work fine on a 10g service without noticeable cpu overhead. It cuts way down on the number of botnet attacks I see.

Here are the scripts I used:
cat /cfg/post_cfg.d/setupfw.sh

#!/bin/sh

ipset -exist create FH_N hash:net
ipset -exist create FH_A hash:ip
ipset -exist create LAN_FH_N hash:net
ipset -exist create LAN_FH_A hash:ip
if iptables -N LOG_DROP >&/dev/null; then
   iptables -A LOG_DROP -m limit --limit 5/min -j LOG --log-prefix "IPTables-Dropped: " --log-level 4
   iptables -A LOG_DROP -j DROP
fi
if !(iptables -L input_wan_rule | grep -q FH_N); then
   iptables -A input_wan_rule -m set --match-set FH_N src -j DROP
   iptables -A input_wan_rule -m set --match-set FH_A src -j DROP
fi
if !(iptables -L forwarding_wan_rule | grep -q FH_N); then
   iptables -A forwarding_wan_rule -m set --match-set FH_N src -j DROP
   iptables -A forwarding_wan_rule -m set --match-set FH_A src -j DROP
fi
if !(iptables -L forwarding_lan_rule | grep -q LAN_FH_N); then
   iptables -A forwarding_lan_rule -m set --match-set LAN_FH_N dst -j LOG_DROP
   iptables -A forwarding_lan_rule -m set --match-set LAN_FH_A dst -j LOG_DROP
fi

#update every 13 minutes
if !(crontab -l | grep -q blockThreats); then
   (crontab -l ; echo '*/13 * * * * /cfg/scripts/blockThreats.sh') |crontab -
fi

cat /cfg/post_cfg.sh

#!/bin/sh
cd $(dirname $0)
for f in post-cfg.d/*.sh; do
    . $f
done

cat /cfg/scripts/fetchBlockLists.sh

#!/bin/sh
# retrieve newer blocklists from firehol, putting them in /tmp
mkdir -p /tmp/blockThreats
for f in $@; do
    tempPath="/tmp/blockThreats/$f.netset"
    url="https://iplists.firehol.org/files/$f.netset"
    if [ -e $tempPath ]; then
        curl -s -z $tempPath -o $tempPath $url
    else
        curl -s -o $tempPath $url
    fi
done

cat /cfg/scripts/doBlockThreats.sh

#!/bin/sh

# doBlockThreads [-private] NET_SET_NAME IP_SET_NAME BLOCK_LIST...
scripts=`dirname $0`

fh_n=$1
shift
fh_a=$1
shift

tempDir=/tmp/blockThreats

ipset -exist create $fh_n hash:net
ipset -exist create $fh_a hash:ip

#echo Getting files...

# merge the files from firehol
for f in $@; do
    if [ -e $tempDir/$f.netset ]; then
        # remove private ranges
        grep -v -e '^192\.168' -e '^172\.16' -e '^10\.' $tempDir/$f.netset >> $tempDir/merged.txt
    fi
done

# sort the merged file and remove any duplicates
sort -u $tempDir/merged.txt > $tempDir/merged_sorted.txt
rm $tempDir/merged.txt

# Output the Network CIDRs from the merged_sorted file
getnetblocks() {
cat <<EOF
# Generated by ipset
-N geotmp nethash --hashsize 1024
EOF
cat $tempDir/merged_sorted.txt|egrep '^[0-9]'|egrep '/' |sed -e "s/^/-A geotmp /"
}


# create the ipset FH_N
#echo Create the Network ipset...
getnetblocks > $tempDir/network_ipset.txt
ipset -! restore < $tempDir/network_ipset.txt
ipset swap geotmp $fh_n
ipset destroy geotmp


# Output the IPs from the merged_sorted file
getaddblocks() {
cat <<EOF
# Generated by ipset
-N geotmp hash:ip --hashsize 1024
EOF
cat $tempDir/merged_sorted.txt|egrep '^[0-9]'|egrep -v '/' |sed -e "s/^/-A geotmp /"
}


# create the ipset FH_A
#echo Create the Address ipset...
getaddblocks > $tempDir/address_ipset.txt
ipset -! restore < $tempDir/address_ipset.txt
ipset swap geotmp $fh_a
ipset destroy geotmp

# cleanup
#echo Cleanup...
rm $tempDir/merged_sorted.txt
rm $tempDir/network_ipset.txt
rm $tempDir/address_ipset.txt
#echo Done, total execution time was $SECONDS seconds

cat /etc/cfg/scripts/blockThreats.sh

#!/bin/sh

scripts=`dirname $0`
$scripts/fetchBlockLists.sh firehol_level1 firehol_level2 firehol_webserver firehol_webclient
$scripts/doBlockThreats.sh FH_N FH_A firehol_level1 firehol_level2 firehol_webserver firehol_webclient
$scripts/doBlockThreats.sh LAN_FH_N LAN_FH_A firehol_level1 firehol_level2 firehol_webclient

6 Likes

Very nicely done!

Wonder if this works on other lists as well :eyes:

As long as the lists are made up of ipv4 addresses and ipv4 netblocks, this script will happily merge them and generate ipsets.
Here’s what firehol maintains: https://github.com/firehol/blocklist-ipsets?tab=readme-ov-file#list-of-ipsets-included.
For outgoing ip blocking, logging is turned on, so you should see in /var/log/messages what ips are being blocked. I tried turning on logging for blocked incoming ips, but there are too many for that to make sense - and it’s not particularly helpful to see them.
You can get counts with -
Outgoing (if > 0 might be worth investigating):

root@route10:~# iptables -vL forwarding_lan_rule
Chain forwarding_lan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG_DROP   all  --  any    any     anywhere             anywhere             match-set LAN_FH_N dst
    0     0 LOG_DROP   all  --  any    any     anywhere             anywhere             match-set LAN_FH_A dst
root@route10:~# 

Incoming, but not forwarded:

root@route10:~# iptables -vL input_wan_rule
Chain input_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 3361  151K DROP       all  --  any    any     anywhere             anywhere             match-set FH_N src
  346 17542 DROP       all  --  any    any     anywhere             anywhere             match-set FH_A src
root@route10:~# 

Incoming, forwarded:

root@route10:~# iptables -vL forwarding_wan_rule
Chain forwarding_wan_rule (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 2447  141K DROP       all  --  any    any     anywhere             anywhere             match-set FH_N src
 5104  301K DROP       all  --  any    any     anywhere             anywhere             match-set FH_A src
root@route10:~#