/ 
App-BlockWebFlooders-0.010
River stage one • 1 direct dependent • 1 total dependent
/ block-web-flooders

NAME

block-web-flooders - Block IP addresses of web flooders using iptables

VERSION

This document describes version 0.010 of block-web-flooders (from Perl distribution App-BlockWebFlooders), released on 2019-01-29.

SYNOPSIS

This script should be run as root/sudo root, because it needs to call the iptables command to add block rules to the firewall.

First of all, create /etc/block-web-flooders.conf that contains something like this:

whitelist_ip = 1.2.3.4
whitelist_ip = ...

Where 1.2.3.4 is the IP address(es) that you are connecting from (you can see this from output of w command), to make sure you yourself don't get blocked. Add more lines/IP address as necessary.

When a flood is happening, try to tail your web access log file:

# tail -f /s/example.com/syslog/https_access.2017-06-07.log

and see the patterns that you can use to discriminate the requests coming from the flooder. Since the IP address is usually random/many, you can see from other patterns e.g. requested URI, user agent. For example, if the suspicious log lines are something like this:

93.186.253.79 - - [07/Jun/2017:00:54:23 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
51.15.41.74 - - [07/Jun/2017:00:54:25 +0000] "POST /heavy2.php HTTP/1.1" 302 - "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.38.149.5 - - [07/Jun/2017:00:54:24 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
93.186.253.79 - - [07/Jun/2017:00:54:24 +0000] "GET /heavy3.php HTTP/1.0" 200 20524 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
51.15.41.74 - - [07/Jun/2017:00:54:25 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.38.149.5 - - [07/Jun/2017:00:54:25 +0000] "GET /heavy3.php HTTP/1.0" 200 20524 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.38.149.5 - - [07/Jun/2017:00:54:25 +0000] "GET /heavy3.php HTTP/1.0" 200 20524 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
93.186.253.79 - - [07/Jun/2017:00:54:26 +0000] "POST /heavy2.php HTTP/1.1" 302 - "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
51.15.41.74 - - [07/Jun/2017:00:54:25 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.36.213.37 - - [07/Jun/2017:00:54:26 +0000] "GET /heavy3.php HTTP/1.0" 200 20524 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.36.213.37 - - [07/Jun/2017:00:54:27 +0000] "POST /heavy2.php HTTP/1.1" 302 - "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.38.149.5 - - [07/Jun/2017:00:54:26 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"
89.36.213.37 - - [07/Jun/2017:00:54:26 +0000] "GET /heavy1.php HTTP/1.0" 200 20633 "-" "Opera/9.80 (Windows NT 6.0; U; en) Presto/2.2.0 Version/10.00"

you can add --has Presto/2.2.0 and --has /heavy since these quite accurately selects the flood requests. If you can add strings which pretty accurately single out the flood requests, you can use a lower threshold speed, e.g. --limit 5 to block IPs which has requested 5 or more in the last 5 minutes. Otherwise, if you do not have any specific --has to single out the flood, you might need to set a higher limit, e.g. --has html --limit 30 --period 60 to block IPs which have requested 30 or more requests in the last minute, or --limit 200 --period 120 to block IPs which have requested 200 or more requests in the last 2 minutes.

Feed the output of the tail command to this script:

# tail -f /s/example.com/syslog/https_access.2017-06-07.log | block-web-flooders \
  --has Presto/2.2.0 --has-pattern '/heavy|/baz' --limit 5

or perhaps:

# tail -f /s/example.com/syslog/https_access.2017-06-07.log | block-web-flooders \
  --limit 200 --period 120

The script will display the top IP addresses and whether an IP is being blocked, along with some statistics:

Blocked IPs this session:  12 | Log lines:  198 | Running for: 2m13s
Top IPs:
  89.36.213.37    (  4)
  89.38.149.5     (  2)
  93.186.253.79   (  2)
  ...
Last messages:
  51.15.41.74 BLOCKED

While this script is running, you might also want to open something like this in another terminal (monitor incoming web requests):

# tail -f /s/example.com/syslog/https_access.2017-06-07.log | grep /heavy

and somethins like this in yet another terminal (monitor system load and number of web server processes, this depends on the web server you use):

# watch 'w | head -n1; echo -n "Blocked IPs total: "; iptables -nL INPUT | wc -l; echo -n "Apache processes: "; ps ax | grep apache | wc -l'

If your webserver is still maxed out by requests, you might want to tweak --limit and --period options and restart the web server.

To see the blocked IP addresses:

# iptables -nL INPUT

As long as the script runs, IP addresses are blocked by default temporarily for 86400 seconds (or, according to the --block-period command-line option or block_period configuration). After that block period is exceeded, the IP is unblocked.

To immediately clear/unblock all the IPs:

# iptables -F INPUT

(this is assuming the default policy of input is ACCEPT; if you have a firewall package installed, please follow the procedure for that firewall.)

To immediately unblock some IPs:

#

DESCRIPTION

This script helps a sysadmin when there is a flood from multiple IP addresses to your website. The script works by reading web access log file, considering lines which match the specified pattern(s), then block the IP address of the requester if the speed of request from that IP exceeds a limit. The blocking is done using firewall (iptables), by default:

# iptables -A INPUT -s <ip-address> -p tcp -m multiport --dports 80,443 -j DROP

To use this script, see the general guide in the Synopsis.

OPTIONS

  • --has=S

  • --has-pattern=REGEX

  • --lacks=S

  • --lacks-pattern=REGEX

  • --limit=N

  • --period=N

  • --whitelist-ip=IP

  • --spanel-site=NAME

  • --dry-run

TODO

Option to customize ports.

Parse timestamps from web access logs so it can also parse past lines.

IPv6 support.

Some interactivity, e.g.: reset counters, unblock some IPs, increase/decrease limit.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/App-BlockWebFlooders.

SOURCE

Source repository is at https://github.com/perlancar/perl-App-BlockWebFlooders.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=App-BlockWebFlooders

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

CSF, https://configserver.com/cp/csf.html, the more generic solution.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2019, 2018, 2017 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.