Spam Filtering with Qmail using Spamdyke

Intro
This HOWTO should give an overview on using spamdyke SMTP filter in front of the mail-mta/netqmail mailserver. It has been clumsily recovered from Google cache. Any corrections or reformatings are appreciated. From the spamdyke website:


 * spamdyke is a filter for monitoring and intercepting SMTP connections between a remote host and a qmail server. Spam is blocked while the remote server (spammer) is still connected; no additional processing or storage is needed. In addition to all of its anti-spam filters, spamdyke also includes a number of features to enhance qmail.

This HOWTO asumes a working mail-mta/netqmail installation with optional net-mail/vpopmail support. If you don't have these pre-conditions fulfilled, please consult the official netqmail/vpopmail Virtual Mail Hosting System Guide or one or more of the following resources to get your environment ready:

Pre-conditions
This howto was created on a "i686-pc-linux-gnu" with the following ebuilds installed:


 * (mailwrapper noauthcram qmail-spp ssl -gencertdaily -highvolume -vanilla)
 * (clearpasswd mysql -ipalias)
 * (clamav passthru per-domain regex spamassassin -attachment -dropmsg -quarantine -received)
 * (maildrop)
 * (authlib berkdb gdbm mysql -debug -fam -ldap -postgres)
 * (berkdb doc ipv6 mysql qmail ssl tools -ldap -postgres -Useflag|sqlite)
 * (bzip2 crypt logrotate nls -mailwrapper -milter -selinux)

Different environments may affect the way this HOWTO should work and can lead to strange effects. If you experience anything in this direction, please leave a note here or on the discussion page.

Installation
If you need/use SMTP-AUTH with your Qmail installation, add the following to :

If the package is masked, add the following to : =mail-filter/spamdyke-3.1.8

Then emerge the package: Emerging spamdyke

The binary is installed to whereas the configuration is in

Download and unpack
Download spamdyke-3.1.6 from the spamdyke releases website to a directory of your choice and upack the tar ball:

wget http://www.spamdyke.org/releases/spamdyke-3.1.6.tgz

tar -xvzpf spamdyke-3.1.6.tgz

Compile
spamdyke does only require the usual compilation procedure in the sub-folder spamdyke/ of the just extracted spamdyke-3.1.6/ directory. cd spamdyke-3.1.6/spamdyke

./configure

make

The ./configure script automatically detects if you need TLS support (by checking for OpenSSL) and compiles spamdyke accordingly. The resulting spamdyke binary is the only file produced and needed.

Install
To install the spamdyke binary to your system copy it over to.

cp spamdyke /usr/local/bin/spamdyke

Configuration
After successfully installing the spamdyke binary, it needs to be integrated into the mail-mta/netqmail setup. The documentation spamdyke-3.1.6/documentation/INSTALL.txt suggests to configure spamdyke directly in /service/qmail-smtpd/run, but to stick with Gentoo style, changes are only performed in /var/qmail/control/conf-smtpd. Getting started

spamdyke needs to be placed before qmail-smtpd receives the incoming mail. It will then decide upon its configuration if an incoming mail should be directly rejected or if it should be passed on to this MTA. To achieve this on Gentoo, the QMAIL_SMTP_PRE variable is the corrects place to hook spamdyke in:

This adds spamdyke right before Qmail accepts an incoming connection. The filter needs to know the hostname of the local machine which is available through the environment variable $HOSTNAME, if you configured it correctly in /etc/conf.d/hostname.

The second parameter is a config file that contains all further configurations. spamdyke can also get all its configuration values from commandline options, but maintaining them in a config file is a) more comfortable and b) it is even faster, as the spamdyke website states:

Basic configuration
The /etc/spamdyke/spamdyke.conf should contain the following lines for testing:

Log level 2 indicates that all errors and info messages should be logged. You should switch that to level 1 after testing, because spamdyke will probably log a loooot of blocked spam. ;) The local-domains setting is needed since spamdyke will reject all emails to other domains automatically, of no SMTP-Auth is provided. The last setting is only neccessary if you use TLS/SSL to secure connections and want spamdyke to inspect such connections, too.

Getting spamdyke to run
To make Qmail use your fresh installed spamdyke, just restart Qmail: /etc/init.d/svscan restart

Options
spamdyke supports a huge lot of options to affect its behaviour. The most important ones for fighting spam are blacklisting options, that allow you to define which mails should be rejected by spamdyke. You get a complete overview on available options by calling spamdyke -h. These are also explained in detail in spamdyke-3.1.6/documentation/README.txt.

In the configuration file you can use every long option without the leading 2 dashes. If a value is expected, just add a = and the value after that.

Recommended options by this article are:

With these options spamdyke will reject most of the connections from dialup networks, since those usually don't have a reverse DNS entry setup. Beside that, email that appears to come from a domain that does not have an MX or even an A DNS entry, will be rejected, which is often the case for spammer domains. This will most probably reduce already a huge lot of your spam, sent by Windoze PCs infected with bots and the like.

SSL / TLS
If your mail-mta/netqmail is compiled with ssl support, spamdyke cannot examine emails send through encrypted connections and can only include the sender IP in its checks. To enable spamdyke to examine the email content (especially the sender and receiver) it needs to use the same SSL certificate than qmail does:

SMTP-Auth
Spamdyke does now support SMTP-Auth out of the box. It makes Qmail serve the authentication and examines the status. If authentication fails, spamdyke will also reject the connection. Blacklisting

The setup this HOWTO is based on makes use of 2 differnt kinds of blacklists:


 * ip-blacklist-file: Allows you to specify a file which contains a list of IP addresses to block.
 * check-dnsrbl: Although a DNSBL lookup is slower than local file updates I'm using several DNSBLs in addition to the local IP based one.

Whitelisting
Whitelisting is the opposite of blacklisting. For all hosts that are found in an IP whitelist all further checks are ignored and email is just forwarded to Qmail. Therefore you should only use this features with hosts for that you can definitly ensure they are not compromised or send spam because of another reason.


 * ip-whitelist-file: The given file contains a newline seperated list of IP addresses.

Tips and tricks
This section provides practical tips and tricks, which are not neccessarily useful for everyone. It is recommended to just read through it and check what you find interesting or to find creative ideas.

NiX spam
To update the NiX-spam blacklist, run the following script every 30 minutes via a CRON job:

NiX spam makes only sense if you are located in Germany or at least western Europe. If you have other IP based blacklists to use, this script should be adjustable to your needs. Remember to create the directory where blacklists should be stored and to make them readble for the user running spamdyke.

DNSBL
For DNSBL I use the following providers:


 * zen.spamhaus.org
 * list.dsbl.org (Status as of 13.07.2008: offline)
 * cbl.abuseat.org (already implemented in the Spamhaus database)
 * spamsources.fabel.dk
 * ix.dnsbl.manitu.net

The last one is the DNSBL server of NiX-spam, which contains more and especially more up2date IP addresses, which might find spammers even if their IP address is not in the local IPBL.