Adding the Spamhaus drop list to MimeDefang on FreeBSD

We have decided to use the Spamhaus drop list on our anti-spam system, which will help us block certain net blocks that have been hijacked by spammers. We wanted to use this list in line with the other features of our Anti-Spam solution which means the ability to control it on a per-domain basis and rejections should be logged into our user accessible logging system. Adding functionality to MimeDefang seemed to be the obvious answer. The lists are downloadable from Spamhaus from these URLs and we have set up a cron task to keep them up to date as follows:

#! /bin/sh

cd /usr/local/etc/mimedefang
fetch http://www.spamhaus.org/drop/edrop.txt
fetch http://www.spamhaus.org/drop/drop.txt

This has been cronned as a weekly task – I’ll put some error checking in later when I’m happy the rest of the system is working. The content of the list looks like this:

; Spamhaus DROP List 01/04/13 - (c) 2013 The Spamhaus Project
; Last-Modified: Wed,  2 Jan 2013 05:55:18 GMT
; Expires: Sat, 05 Jan 2013 10:20:18 GMT
5.34.242.0/24 ; SBL154880
5.62.128.0/17 ; SBL167294
5.72.0.0/14 ; SBL167293
14.192.0.0/19 ; SBL123577
...continues

So in order to manipulate the list we can use the perl module Net::CIDR which is easily installed on FreeBSD like this:

cd /usr/ports/net-mgmt/p5-Net-CIDR/
make install

Now, to the code. I’l be modifying the mimedefang-filter file to do this, but I’ll do a quick test first to make sure its all going to work.

#! /usr/bin/perl -w

use Net::CIDR;

my $ip = '205.189.71.1';

open DROP, "/usr/local/etc/mimedefang/drop.txt" or die "cannot open drop.txt ($!)";
open EDROP, "/usr/local/etc/mimedefang/edrop.txt" or die "cannot open edrop.txt ($!)";
my @test;
while (<DROP>){
s/( |^);.*$//;
if (! /^$/){push (@test,$_)}
}

while (<EDROP>){
s/( |^);.*$//;
if (! /^$/){push (@test,$_)}
}

if (Net::CIDR::cidrlookup($ip, @test)){
print "Gotcha!\n";
}else{
print "Not Found\n";
}

I thought that was a lot of work for the server to do on every email so I decided to re-write the bash scrip with Perl and added some error checking too! This new script produces a nice file with just the ip addresses in CIDR (x.x.x.x/xx) format and both lists in one file. The idea is to do as much of the processing onece per week to reduce the load on a mail by mail basis. Here is the new script:

#! /usr/bin/perl -w

use LWP::Simple;
my $drop = get "http://www.spamhaus.org/drop/drop.txt" or die "Could not download drop.txt : $!";
my $edrop = get "http://www.spamhaus.org/drop/edrop.txt" or die "Could not download edrop.txt : $!";
my @drop = split(/\n/,$drop);
my @edrop = split(/\n/,$edrop);

open DROP, "> /usr/local/etc/mimedefang/dropfile.txt" or die "Could not open file for writing : $!";

for (@drop, @edrop){
if (! /^\s*($|;)/){
s/\s*;.*//;
print DROP "$_\n";
}
}

close DROP;

Just as a test I tried running the test script shown below with the file stored on RAMDisk or on a normal disk and it seemed to show no significant improvement in speed so I’m leaving the file on the normal disk. here is my new test file using the new download script:

#! /usr/bin/perl -w

use Net::CIDR;
my $ip = '205.189.71.1';
open DROP, "/usr/local/etc/mimedefang/dropfile.txt" or die "cannot open dropfile.txt ($!)";
while (<DROP>){
if (Net::CIDR::cidrlookup($ip, $_)){
print "Gotcha!\n";
}
}

So now we just need to cut that into the mimedefang-filter to make it work. I put my code snippet (below) within the sub filter_recipient as the first task, deciding that this was ‘less expensive’ than checking white/blacklists which require MySQL lookups. Here’s the snippet:

use Net::CIDR;
open DROP, "/usr/local/etc/mimedefang/dropfile.txt" or warn "cannot open dropfile.txt ($!)";
while (<DROP>){
if (Net::CIDR::cidrlookup($ip, $_)){
&logger ("SPAMHAUS DROP LIST", );
return ('REJECT', "Your ip address ($ip) is listed in Spamhaus drop list");
}
}
close DROP;