Why not adopt me?
NAME
SyslogScan::Daemon::SpamDetector::BlockList - maintain an IP-based blocklist of spam sources
SYNOPSIS
plugin SyslogScan::Daemon::SpamDetector as sd_
sd_plugin SyslogScan::Daemon::SpamDetector::BlockList
debug 3
track_dbi_dsn 'DBI:mysql:database=mailstats;host=localhost'
track_dbi_user 'username'
track_dbi_pass 'passwd'
block_dbi_dsn 'DBI:mysql:database=iimaildb;host=blockhost'
block_dbi_user 'username'
block_dbi_pass 'passwd'
minblock 0.0416666666666667 # minimum block length (days)
maxblock 1.0 # maximum block length (days)
blockmult 1.2 # amount to grow/shrink the block
shrinktime 0.25 # grow the block length if spammed again in less than this (days)
growtime 0.04 # amount of times w/o a spam to shrink block length (days)
firstblock 10 # number of spams required to trigger first blocking
cleanup 86400 # clean up tables, remove blocks (seconds)
blockmemory 8 # keep history of previous blocks (days)
spammemory 8 # keep history of spams sent (days)
recipweight .33333333333333 # how much does an extra recipient count towards being enough spam to block
blockcommand ''
repeatblocks 50 # times blocked to get worse treatment
repeatoffender 250 # count of spams to get worse treatment
rpmult 4 # how much more/less time repeat offenders get to grow shrink their block time
fastspammer 20 # number of spams sent while blocked to get penealty
fastspammermult 1.5 # blocktime multipe for fastspammers
DESCRIPTION
Track and react to the spam noticed by SyslogScan::Daemon::SpamDetector.
Track the spam in an SQL database. Build a blocklist.
Block hosts for a bit. If they keep spamming, block them for longer periods.
We reccomend that this module not be combined with SyslogScan::Daemon::SpamDetector::SpamAssassin becuase it will end up blocking forwarded mail.
CONFIGURATION PARAMETERS
The following configuration parameters are supported:
- debug
-
Debugging 0 (off) to 5 (verbose).
- minblock
-
The minimum amount of time to block an IP address in days. (Default: about an hour)
- maxblock
-
The maximum amount of time to block an IP address in days. (Default: 1)
- blockmult
-
If a site is being re-blocked within the
growtime
window, multiply the block time by this amount. If a site is being re-blocked outside of theshrinktime
window then divide the block time by this amount. (Default: 1.2) - growtime
-
If a site sends more spam and thus needs re-blocking, punish with a longer block if it's needing re-blocking before this amount of time has passed. In days. (Default: about one hour)
- shrinktime
-
If a site sends more spam and thus needs re-blocking but it has been longer than this amount of time since it was last blocked, then give it a shorter block than it had before. In days. (Default: .25)
- firstblock
-
Don't block a site until this many spam have been recevied. (Default: 10)
- block_dbi_dsn =item block_dbi_user =item block_dbi_pass
-
Use this database for the IP block list. The table is small.
- track_dbi_dsn =item track_dbi_user =item track_dbi_pass
-
Use this database for tracking incoming spam. The table is large.
- cleanup
-
Perform cleanup of old on the database this often. Seconds. (Default: 43200 - 12 hours)
- blockmemory
-
Remember inactive blocks for this long. Days. (Default: 4).
- spammemory
-
Remember incoming spam for this long. Days. (Default: 10).
- blockcommand
-
Run this command when a block is created. (No default)
- recipweight
-
When the same message is sent to multiple recipients, count the subsequent deliveries as this much of a spam. (Default: .333333333333)
- repeatblocks
-
When an IP address has been been blocked this man times treat it specially: instead of multiplying block times by
blockmult
, multiply them byrpmult
. - repeatoffender
-
When an IP address has sent this many spams, treat it specially: instead of multiplying block times by
blockmult
, multiply them byrpmult
. - rpmult
-
For espeically bad sources, instead of multiplying block times by
blockmult
, multiply them byrpmult
. (Default: 4) - fastspammer
-
If this many spams are sent while being blocked (presumably because the block doesn't cover the entire network), then multiply the block time by
fastspammermult
. - fastspammermult
-
Block multiplier penealty for fast spammers. (Default: 1.5)
- cleanupfirst
-
Clean out old entries from the database tables at startup? (Default: 1)
TABLE CREATION
SQL tables are not automatically created. Here are examples for Mysql5:
CREATE TABLE spam_received (
ip VARCHAR(16),
spamtime DATETIME,
messageid TEXT,
recipients INT,
host TEXT,
score TEXT,
matched TEXT,
INDEX ip_index(ip)
) TYPE=MyISAM;
CREATE TABLE block_history (
ip VARCHAR(16),
blockstart DATETIME,
blockend DATETIME,
blockcount INT,
spamcount INT,
messageid TEXT,
countdays INT,
PRIMARY KEY (ip),
INDEX done(blockend)
) TYPE=MyISAM;
Replace "Los Angeles" with a city in your time zone.
drop view blocking;
CREATE VIEW blocking AS
SELECT ip,
CONCAT("450 blocked for ",
SUBTIME(TIMEDIFF(blockend,blockstart),
SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))),
" until ", ADDTIME(blockend, timediff(now(), utc_timestamp())),
"-Los Angeles time due to ", spamcount, " spams in ", countdays,
" days, for example <", messageid, ">") AS message,
SUBTIME(TIMEDIFF(blockend,blockstart),
SEC_TO_TIME(SECOND(TIMEDIFF(blockend,blockstart)))) AS blocktime,
case when blockend > utc_timestamp() then 1 else 0 end AS active,
ADDTIME(blockend, timediff(now(), utc_timestamp())) as blockend_local,
spamcount
FROM block_history;
USEFUL QUERIES
To watch what's going on with the block list, the following queries are helpful.
select ip, COUNT(*), SUM(recipients) from spam_received
group by ip order by COUNT(*) desc limit 25;
select COUNT(*), SUM(recipients) from spam_received;
select active, count(*), AVG(blocktime), AVG(spamcount), MAX(spamcount)
from blocking group by active;
select ip, message from blocking where active = 1 order by spamcount;
select spamcount, COUNT(*), AVG(blocktime), AVG(spamcount)
from blocking where active = 1 group by spamcount order by spamcount;
select blocktime, count(*), MIN(spamcount), AVG(spamcount), MAX(spamcount)
from blocking where active = 1 group by blocktime order by blocktime;
select ip, blocktime, spamcount, active
from blocking
order by spamcount desc
limit 25;
select count(*), SUM(active) from blocking where spamcount > 250;
select truncate(spamcount,-1), COUNT(*), AVG(blocktime), AVG(spamcount)
from blocking where active = 1 group by truncate(spamcount,-1) order by truncate(spamcount,-1);
POSTFIX CONFIG
To configure Postfix to block using this block list, include a line like:
smtpd_client_restrictions = check_client_access mysql:/etc/postfix/mysql_spam_senders.cf
Then include a query in the referenced file:
hosts = localhost maildb
dbname = spamdb
user = dbuser
password = dbpass
query = SELECT message FROM blocking WHERE ip = '%s' AND active = 1
If your postfix doesn't have SQL support or you're using a less capable mailer, you can dump the block list into a file:
mysql -N -B --host=mx --database=spamdb --user=dbuser \
-e 'select ip, message from blocking where active = 1' \
> file
DNS BLOCKLIST
One way to turn this blocklist into a DNS-based blocklist that can be used like other blocklists is to use rbldnsd(8).
Set up a SyslogScan::Daemon::Periodic job to rebuild the blocklist:
plugin SyslogScan::Daemon::Periodic
debug 1
period 120
command '/usr/local/bin/update_blocklist'
The update_blocklist
script I use is:
#!/bin/sh -e
SUB=1171308796
SECONDS=`date +%s`
SEQN=`expr $SECONDS - $SUB`
DIR="/var/lib/rbldns"
ZONE="blocked"
sed "s/SERIAL/$SEQN/" < $DIR/$ZONE.head > $DIR/$ZONE.new
mysql -N -B --host=MYDBHOST --database=MYDB --user=MYDBUSER \
-e 'select ip, concat(":1:",message) \
from blocking where active = 1' \
| sed 's/:450 /:/' \
>> $DIR/$ZONE.new
mv $DIR/$ZONE.new $DIR/$ZONE
The /var/lib/rbldns/blocked.head
file I use:
$SOA 1800 ns.idiom.com. muir.idiom.com. SERIAL 300 60 3600 300
$NS 1800 MYHOSTNAME
$TTL 120
I start rbldnsd(8) with:
/usr/sbin/rbldnsd -p /var/run/rbldnsd.pid \
-r/var/lib/rbldns -4 -bMYIPADDRESS/53 \
spammers.MYDOMAIN:ip4set:blocked
Then, to use it with postfix(1), I set up a rbl_reply_maps so that I can give a temporary failure.
smtpd_client_restrictions =
permit_mynetworks,
reject_unauth_pipelining,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client spammers.MYDOMAIN
rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
My rbl_rply_maps file:
cbl.abuseat.org $rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason}
spammers.MYDOMAIN 450 4.7.1 Service unavailable; $rbl_class [$rbl_what] $rbl_reason
SEE ALSO
SyslogScan::Daemon::SpamDetector
THANK THE AUTHOR
Hire the author to do some perl programming on your behalf!
LICENSE
Copyright(C) 2006 David Muir Sharnoff <muir@idiom.com>. This module may be used and distributed on the same terms as Perl itself.