Fairly-Secure Anti-SPAM Gateway Using OpenBSD, Postfix, Amavisd-new, SpamAssassin, Razor and DCC
Note: This is an old version of my directions. The system this document describes was designed before the new Postfix Before-Queue Filtering proxy system, which was released 4/23/2004. It will provide much better results with less overhead than the system described here. It is suggested that you use the directions listed here instead of this document.
By Scott Vintinner
Last Edited 3/22/2004 3:30 PM EDT (Changelog at bottom)
Latest version of this document always available at http://www.flakshack.com/anti-spam
Read and post comments here (note
that you don't need to create an account to post).
Index
What is this document?
Where to find help
Notes and conventions
Provide firewall access
Installing OpenBSD
Configuring OpenBSD
Patching/Upgrading OpenBSD
Installing OpenBSD Ports
Removing Sendmail
Add User Accounts
Installing and Configuring Postfix
Installing Needed Perl Modules
Installing and Configuring SpamAssassin
Installing Razor
Installing DCC
Installing and configuring Amavisd-new
Installing Postfix Email Reporting
What if I want to disable Amavisd-new/SpamAssassin?
Keeping the mail queue clean
Improving Performance
Performance Expectations
Configuring for Multiple Domains
Notes about running under other OS
Adding Anti-Virus Support
Download My Scripts
Changelog
Legal Stuff
What
is this document?
This document describes how to setup a spam-blocking email gateway based on
open source and freely available software. This procedure is designed for a
small to medium sized company with a single domain (multiple domains are possible...just
not described here). I will describe how to setup a new computer that is meant
to run on your network's DMZ in between the Internet and a corporate email server
like Lotus Notes or Microsoft Exchange.
This entire procedure has been developed with security as a primary focus.
The operating system is OpenBSD (www.openbsd.org),
which is a "Secure by Default" system with an amazing track record
for security. The email MTA is Postfix (www.postfix.org)
which also has a good record for security and is the easier of the 2 main competitors
to the troubled sendmail program. Amavisd-new
(www.ijs.si/software/amavisd) is the main filter which processes email from
postfix and ensures that we don't lose any mail. Amavisd-new is an huge improvement
over the original amavis which was a simple virus scanner, and I think it is
the best way of implementing SpamAssassin
(www.spamassassin.org). SpamAssassin is the main anti-spam component which
works by comparing messages to a ruleset and by using a statistical analysis
that is custom built based on your email. In addition to the SpamAssassin spam
detection software, we will be using 2 online SPAM databases: DCC
(www.rhyolite.com/anti-spam/dcc) and Vipul's
Razor (razor.sourceforge.net). These databases work by comparing hashes
of our email messages with hashes of known spam. As a final security precaution,
we will run all network processes in a restricted-user/chroot environment,so
if an attacker were able to compromise one of the modules, the amount of damage
they could do would be seriously limited.
There are many different ways to customize the handling of SPAM with this solution.
At my company we debated the merits of a number of different methods. We considered
using the ability of amavisd to add extensions on spam emails (so a spam email
to john@domain.com becomes john+spam@domain.com). Under this plan, we would
setup a public folder on Exchange for each user and assign it that +spam email
address. This way the spam would not take up space in their mailbox, and similar
to Hotmail, the user could occasionally check the folder for false-positives.
We could then write a VB program to clean out those public folders every 90
days. We also considered showing users how to create rules in Outlook to detect
the X-SPAM header and automatically move the email to a subfolder that they
would again check less frequently, or directly to their Deleted Items. In the
end, we decided that a spam bouncing system made the most sense. Every other
method was only a minor improvement over receiving the spam directly in the
first place. By bouncing the spam back to the sender, we put the ball back in
the sender's court by letting the sender know their message was rejected.
Since false positives were such a big concern, we implemented this process in
2 stages: a learning stage followed by a blocking stage. For the first 45 days
we ran the system in learning mode only. In other words, the system didn't actually
block any spam, it just tagged the spam. This 45 day period gave SpamAssassin
time to become more accurate by building up its Bayes database and auto-whitelist.
After this 45 day period, when we were sure that our false positive rate had
been reduced as low as it could go, we switched amavisd over to Bounce mode.
In this mode, the system rejects any message it diagnoses as spam. The sender
of the message receives a nice email from the system telling them why their
message was rejected (including the SpamAssassin score distribution) and how
they can get on our whitelist. Since spammers rarely use real addresses, few
if any of these bounce messages will be delivered to spammers, so we also include
a script to delete these undeliverable messages so they don't clog up the queue.
In the end, you'll have an anti-spam solution that requires no effort by your
end users. There are no spam-folders for them to check; the spam just stops.
In the rare event of a false positive, the sender receives a nice message telling
them how to get their message delivered. This document describes how to setup
the system we used.
I've been receiving an average of 5 new emails per day from people asking me
to help debug their setups. In nearly all of these cases, the problem is due
to a typo in the configuration files or some minor fluke caused by using a slightly
different version of one of the components. Unfortunately I no longer have the
time to answer everyone's questions personally.
The best place to go for help is the amavisd mailing list (https://lists.sourceforge.net/lists/listinfo/amavis-user).
Please search through the archive (http://marc.theaimsgroup.com/?l=amavis-user)
for answers to your problem before posting a question. Mark Martinec (the main
amavisd guy) does a very good job of answering most questions quickly, so please
don't waste his time. Do some research first and try to ask good questions.
I also subscribe to the list, so when posting your question be sure to mention
that you are using OpenBSD. Also please do not email huge copies of your config
files or debug output unless asked to do so.
If your question is specific to SpamAssassin, DCC, Razor, or Postfix, please refer to their respective home pages (listed above) for more assistance.
If you have a suggestion, or find an error you think I should fix, please email me at scottv@rbh.com.
Notes
and Conventions
This document is not really meant for the unix newbie. If you
have never worked with linux or unix before, you may experience some difficulty
with these directions. Specifically, I expect that you will know how to use
vi to edit files (here's a vi
cheat sheet). (Actually I've been informed that you can set the EDITOR environment
variable to get OpenBSD to use whatever editor you want for vipw and crontab
-e, so I guess you can use whatever editor you want.)
Also, you'll want to grab a copy of PuTTY
and WinSCP. I'm
not a big fan of installing X with KDE or GNOME on servers, especially when
the server is open to the Internet. These 2 utilities will help you configure
your server remotely without the need for a GUI running on the server itself.
PuTTY is an awesome SSH client that you can use to talk to your OpenBSD server
over a secure link. It has 2 great features: anything you select with your
mouse
is copied to the Windows clipboard, and right-clicking the mouse anywhere in
the window will paste the contents of the clipboard. It works great in insert
mode in vi to paste hundreds of lines. My servers are rack-mounts, so usually
as soon as I get the network running on the OS, I head back to my PC and pull
up PuTTY. WinSCP is a gui file browser for SSH. It will let you easily copy
files back and forth between your Windows PC and the OpenBSD server.
Text listed in blue are commands that you type
into the console. Text listed in red are edits
in text files. Text listed in gray tables are full files. The #
sign followed by a blue command indicates a shell
prompt (so don't type it). The # sign in configuration files or text edits is
used as a comment block, and may be included in the actual files without harming
anything.
I really dislike configuration files that include page after page of commented
instructions. I know it is really the Open Source configuration "thing"
to have fifty lines of comments to explain a single configuration option (as
anyone that has ever run apache will tell you). While that may be an easy way
for the authors to explain their config files, it is definitely not a good way
for administrators to work. By taking the comments out of the amavisd.conf file,
it went from a twenty page monster down to more manageable page and a half.
If you want my advice, print out the original conf file as a reference, then
remove all the comments except the absolute essentials.
I have received many many emails from people telling me that I'm going straight
to hell for not using SUDO. As
you look through my directions, you will see that the directions have you logged
in as root during this entire procedure. The "best-practices" for OpenBSD say
that you should login as an unpriviledged account to do most of the work and
use the sudo command
whenever
root access
is required.
I'll be the first to admit that I'm not conscientious (anal) enough to follow
those guidelines 100% of the time. I certainly suggest that you read up on
the sudo
command if you are unfamiliar
with it.
Personally
I
think
there
is
a time
and
a place
for
using
sudo and
an efficient administrator knows
the
difference.
One last note: obviously these directions require some customization to fit
into your network. Specifically you'll want to replace the sample server
names
and IP addresses with your own. Here's a little diagram of what we'll be
setting up:
For our examples, our internal network numbering uses the 10.0.0.0/8 private domain range. The 10.1.0.0/16 network is our main network. The 10.2.0.0/16 network is our DMZ network. Our pretend public internet class C is 65.0.0.0/24. Additionally, the server we will be setting up will be called mta1 in these directions. You can call your server whatever you want, just replace it with your name where ever you see it. For simplicity, we'll pretend that our company's domain name is domain.com.
Hopefully you have a firewall at your company. Here's a list of what traffic you will need to allow through your firewall to make all of this work.
| Source |
Destination |
Type |
Port |
Description |
| 10.2.1.50 |
10.1.1.50 and any External |
TCP |
25 |
Outgoing SMTP |
| 10.1.1.50 |
10.2.1.50 |
TCP |
25 |
SMTP from Exchange to our MTA |
| any External | 10.2.1.50 | TCP | 25 | Incoming SMTP |
| 10.2.1.50 |
any External |
UDP |
6277 |
Outgoing from our MTA to the DCC servers |
| any External | 10.2.1.50 | UDP | 6277 | Incoming from DCC Servers to our MTA |
| 10.2.1.50 |
any External |
TCP |
2703 |
Outgoing from our MTA to the Razor servers |
| 10.2.1.50 | any External | TCP | 7 | Outgoing ping from our MTA to the Razor servers |
| 10.2.1.50 | any External | TCP | 22 | Outgoing SSH (used to update our source code from the
OpenBSD servers using CVS) |
| 10.2.1.50 | any External | TCP | 21 | Outgoing FTP (so we can download files that we need) |
| 10.2.1.50 | any External | TCP | 80 | Outgoing HTTP (also so we can download files) |
Additionally mta1
needs to have access to a DNS server. In my configuration the DNS is on the
DMZ so no extra firewall rules were required.
Installing
OpenBSD
The OpenBSD
FAQ Chapter 4 provides an excellent step by step explanation of how to install
the latest version of OpenBSD, so I won't repeat it all here. I'll just give
some minor tips:
The great folks over at OpenBSD release 2 new sub-versions a year. Unfortunately
this means that in between those releases, the only way to install patches is
to apply them yourself. The OpenBSD
errata page lists patches that have been released for OpenBSD. There are
2 ways to do this. The first involves download each of the 14 patch files then
reading and following the directions in each (which usually involves re-compiling
some part of OpenBSD). The second way involves using CVS to automatically download
and update the source tree, then recompiling. I'll discuss the second way, since
it is also how you can upgrade OpenBSD from one version to the next. You might
as well learn how to do this now, rather than 1 year from now when the system
is in production and you need to upgrade it to fix some security hole.
If you are upgrading from one verison of OpenBSD to another (like from version
3.3-current to version 3.4), you will want to read the Upgrading
mini-FAQ. (Note that the binary type changed in 3.3-current from a.out to
ELF, so you can't upgrade 3.3 or earlier systems. Better to just reinstall.
)It describes some extra steps you should take to upgrade your system from older
versions. Most of these directions are identical to the ones listed in that
document.
The best way to install most software for OpenBSD is to use the Ports collection.
The Ports collection is available on your OpenBSD CDROM, or from any of the
FTP mirrors. When unzipped, it contains a number of files that tell the operating
system where to go to download over a thousand programs for OpenBSD, along with
directions on how to compile them. The limitation of the ports collection is
that it does not include the latest and greatest versions of all the software,
so while we will us it to install a bunch of utilities, we'll install the main
programs directly from the source. For more detailed information on how to use
the Ports tree, or how to download it using CVS, see the Ports
and Packages mini-faq.
The first step is to get the ports.tar.gz file from the CD-ROM and copy it to
/usr
# mount
/dev/cd0a /mnt
# cp
/mnt/3.3/ports.tar.gz /usr
# umount
/mnt
#
cd /usr
# tar
-zxvf ports.tar.gz
Once you extract the file, you will find a whole tree of files under /usr/ports.
The tree is divided into sections which categorize the different software packages
available. To install a particular software package, all you need to do is change
into the directory of the software package and run make install:
# cd /usr/ports/archivers/unzip
# make install
The system will automatically download the source code, compile the program,
generate a "package", and install the software. The cool thing is
that the software is installed as a package...meaning you can easily remove
it using the pkg_ commands. For example, to list all installed packages, type
# pkg_info. To remove
a package type # pkg_delete
<packagename>.
So now you need to install all of the programs listed below. As mentioned, all
you need to do is change to each of the following directories and run make install
in each.
| Package Source Directory | Description |
| /usr/ports/archivers/unzip | A file used by amavisd-new when scanning MIME attachments. |
| /usr/ports/archivers/unrar | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/unace | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/unarj | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/arc | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/bzip2 | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/lha | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/archivers/zoo | A file used by amavisd-new when scanning MIME attachments |
| /usr/ports/sysutils/mergemaster | A utility that makes upgrading the OS easier |
In my experience, the package system usually works without a hitch. If you run into any problems, check the output for any error messages. Generally the only problem you may run into is with mismatched packages...in other words, say you install one package from OpenBSD 3.3, then a year from now, you try to install a package using the ports collection from OpenBSD 3.4. Usually when you upgrade, you will want to remove all the packages and reinstall the versions from that version of OpenBSD.
NOTE: these directions assume that this is a new server that is not currently being used. If your server was previously running sendmail or some other MTA, you will want to check the postfix INSTALL file for better instructions on migrating to postfix.
Add User AccountsWhen possible, you never want to run anything as root. The root account has
unrestricted access to everything, so if there is a bug or security hole in
anything you run as root, it could potentially compromise your entire system.
To further tighten the security of our system, we will run different modules
as different user accounts.
Use vipw to edit the password file. You could also use the adduser or useradd
commands, but this is the easiest way for us. Go to the bottom of the file and
add the following lines:
postfix:*:2000:2000::0:0:Postfix
Mail Daemon:/var/empty:/sbin/nologin
amavisd:*:3000:3000::0:0:Amavis Mail Scanner Daemon:/var/amavisd:/sbin/nologin
Next we want to add the group accounts for our new users. To do this, edit
the /etc/group file. Add the following lines (preferrably in numeric order with
any other groups in the list):
postfix:*:2000:
postdrop:*:2001:
amavisd:*:3000:
Here's a quick
explanation of what we'll use the new accounts for: The postfix account is
the account that postfix will run under, and likewise the amavisd account is
the
account that the amavisd program will run under. Running these programs in
non-root accounts is good for security, since if an attacker compromises either
program,
they are still limited by those accounts.
BTW, the * in the first section is in place of where a normal password would
be. The star tells the operating system that the user cannot login. The /sbin/nologin
is the user's shell that is launched when they login...which in this case is
another indicator to the operating system that these users shouldn't be able
to login.
You can install Postfix from the OpenBSD port collection using the same methods described above for the various archivers. The postfix port is located at /usr/ports/mail/postfix. You may want to double-check which version you are getting with that method. Below I describe how to install Postfix directly from the source. As I mentioned before, I prefer using ports for minor parts of the system and using the source for the major parts. This is just a personal preference.
# queue_directory = /var/spool/postfix mail_owner = postfix myhostname = mta1.domain.com local_recipient_maps = smtpd_banner = $myhostname ESMTP transport_maps = hash:/etc/postfix/transport smtpd_recipient_restrictions
= permit_mynetworks, reject_unauth_destination, reject_non_fqdn_recipient notify_classes = 2bounce,policy,protocol,resource,software #content_filter = smtp-amavis:[127.0.0.1]:10024 # Install Time Configuration |
| Explanation of Postfix configuration settings: | |
| queue_directory | Postfix's work directory. Where all the mail will be temporarily stored until it is delivered. |
| daemon_directory | Specifies the location of all the postfix programs. |
| mail_owner | Specifies the user account that will own the mail queues. |
| myhostname | The name of this computer including the domain part. This is used when adding received by headers in email messages. |
| mydomain | This specifies the domain of this current computer. |
| myorigin | This name is added to locally originating email. So if you sent yourself a message from root, it would appear to come from root@mta1.domain.com. |
| mydestination | This setting tells postfix what domains it will accept email for. Please note this should not be used for virtual domains, or for backup MX hosts. In our case, we set it so it would receive mail for something@mta1.domain.com, something@localhost.domain.com, something@domain.com, and something@mail.domain.com |
| mynetworks | This setting tells postfix what networks it should consider local. In other words, computers connecting from any of these networks will be able to relay mail, etc. In our case, we put 127.0.0.0 (for localhost), 65.0.0.0 (for any other computers on our external network), and 10.0.0.0 (for any internal computer). |
| local_recipient_maps | This setting tells postfix where to find the names of local users to accept mail for. We just want to leave this blank (note that removing it all together will cause errors). |
| biff | This setting tells postfix not to use the biff program to let local users know that they have new email. |
| empty_address_recipient | This setting is the destination for undeliverable mail from <> |
| queue_minfree | This setting tells postfix not to accept any messages for delivery if there are less than 8 megs of disk space available. |
| message_size_limit | This sets the maximum size of a message. Messages larger than 20 megs will be rejected. You can increase this based on your own server requirements. |
| mailbox_size_limit | This sets the maximum size of local mailbox files. We set it to 100 megs, although it should never reach this high because our only local mailboxes are spam and notspam |
| smtpd_banner | This is the banner that is displayed to connecting computers. It is a good security practice to give as little information as possible. I've included just the essentials. |
| transport_maps | This setting tells postfix where to find the transport information. The transport file is where we tell Postfix where to route certain mail. In our case, this file is where we tell Postfix that mail for domain.com should be delivered to our exchange server. |
| local_transport | This setting tells postfix that all local mail should be delivered using the local delivery agent. |
| smtpd_helo_restrictions, smtpd_sender_restrictions, smtpd_recipient_restrictions | These settings are used to deny access to postfix based on the HELO command, the sender, or the recipient. The recipient restrictions settings are used to prevent our mail server from being used as an open relay. As configured, the helo and sender restrictions are left open. If you want to play around with these you can, just look them up in the postfix documentation. Normally these settings can be used to help block SPAM. Unfortunately they depend on the assumption that all legitimate senders have their systems correctly configured. In my experience this is never the case, so these settings are more trouble than they are worth. Like the RBL lists, I found that enabling these settings meant that I spent too much time teaching other mail administrators how to correctly configure their systems (Note that we'll use Realtime Blackhole lists (RBLs) in SpamAssassin, not in Postfix because there positive hits will simply result in higher spam scores.) |
| content_filter | This is where we tell postfix to filter all email through our amavisd-new mail filter. |
| notify_classes | This setting tells postfix to send all sorts of notifications to the
postmaster email account. Here's a list of the available options: bounce: Send the postmaster copies of the headers of bounced mail. 2bounce: Send undeliverable bounced mail to the postmaster. delay: Send the postmaster copies of the headers of delayed mail. policy: Send the postmaster a transcript of the entire SMTP session when a client request was rejected because of (UCE) policy. protocol: Send the postmaster a transcript of the entire SMTP session in case of client or server protocol errors. resource: Inform the postmaster of mail not delivered due to resource problems. software: Inform the postmaster of mail not delivered due to software problems. |
| Install Time Configuration | This section holds the settings we used when we installed postfix. We keep them in the config file so that future upgrades will be easier. |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Both amavisd-new and SpamAssassin are written in perl and have a number of
other perl modules as dependencies. Fortunately perl has a built-in way to download
and install these modules. To start the perl command environment use the command:
# perl
-MCPAN -e shell
This command will popup a little cpan> prompt where you can
enter commands. To install a module, type install followed by the module
name (ex. install MIME::Words ). If you need help,
type help.
Here is a list of the modules required. Note that same may return saying they
are up to date, so just move on to the next one.
Occasionally you will be prompted with the following message:
---- Unsatisfied dependencies detected
during [G/GA/GAAS/Somefile.tar.gz] -----
Name of Dependency
Shall I follow them and prepend them to the queue
of modules we are processing right now? [yes]
This is just telling you that the module you are installing needs some other
module to work properly. It is asking your permission to go ahead and install
it first. Go ahead and press enter to accept the default of yes.
Similarly you will be prompted with a number of other questions during this install process. Just press enter to accept the defaults on most of them.
If you run into any errors, try to read the error to figure out what you should
do. In some cases you may get an error when a test that is run as part of the
install fails. To install anyway use the force install
modulename command. For more assistance with cpan and perl go
to www.cpan.org.
Once everything is installed type 'q' to quit.
rewrite_subject 1 use_bayes 1 |
| Explanation of SpamAssassin configuration settings: | |
| rewrite_subject | (1|0) Tells SpamAssassin to change the subject on SPAM messages to include the subject_tag |
| report_safe | (0|1|2) This setting configures how to handle SPAM. A setting of 0 puts the SpamAssassin report into the headers. A setting of 1 puts it in the main email and attaches the original email as an attachment. Setting 2 is similar to setting 1, plus it changes the type of attachment to text/plain (as a security measure). NOTE: this setting has no affect when spamassassin is run through amavisd, so just leave it set to 0. |
| use_terse_report | (0|1) Setting this to 0 gives the normal length explanation of why the message was considered SPAM. Setting it to 1 gives a shorter report. (Note that this report only appears if you change the report_safe setting, or if you configure blocking like we will do...in which case the sender gets this report). |
| use_bayes | (0|1) This setting turns the Bayesean Learning on or off. In our case we want it on. |
| bayes_path | (path) Location of systemwide bayes database. We need to set this so that the root account can run the sa-learn program to update the bayes database normally used by the amavisd user. |
| auto_learn | (0|1) If turned on (1), this will cause SA to automatically add very SPAM or very not-SPAM messages to the Baysean statistics database. |
| skip_rbl_checks | (0|1) Definitely turn this off (0). Unlike using the Blackhole lists from within Postfix, using them in SpamAssassin allows you to selectively use the RBLs. For example, if you have a client that has an open relay and is unwilling to change it, you can simply add their domain to your whitelist, while still being able to use the RBLs for everyone else. |
| use_razor2 | (0|1) Tells SA that we want to use Razor version 2 |
| use_DCC | (0|1) Tells SA that we want to use DCC (Distributed Checksum Clearinghouse) |
| use_pyzor | (0|1) Tells SA that we don't want to use Pyzor (since we won't be installing it). |
| dcc_add_header | (0|1) Tells SA to add a header from DCC containing statistics about the message |
| dns_available | (yes|test|no) Normally SA tests to see if it has access to a DNS server to do lookups. Since I know my server has access to DNS, I tell it to skip this test. It saves on initial amavis startup time. |
| LOCAL_RCVD rules | The last lines header, describe and score are used to prevent my outgoing
mail from being tested for spam. If you're like me, your users would
be upset if their mail was tagged as spam before a client read it. This
rule basically checks the header for the Received from: lines
showing the message route. You will need to customize this rule to fit your system. To do this, send a message to your test hotmail account (or some account outside of your system). Check the headers for lines that look like this: Received: from mta1.domain.com (mta1.domain.com [64.132.107.5]) by law122.ms.hotmail.com (8.12.6/8.12.2) with ESMTP id h3EI48pL002768 for <forge@hotmail.com>; Mon, 14 Apr 2003 14:04:08 -0400 (EDT) Received: from exchange.domain.com (exchange.domain.com [10.1.1.50]) by mta1.domain.com (Postfix) with ESMTP id F3B7117EFD for <forge@hotmail.com>; Mon, 14 Apr 2003 14:04:07 -0400 (EDT) Received: by exchange.domain.com with Internet Mail Service (5.5.2653.19) id <GKFZ3XPK>; Mon, 14 Apr 2003 14:04:03 -0400 The rule is a standard SpamAssassin rule and uses Regular Expression syntax. To explain it in regular terms, it looks for *.domain.com (*[*]) on the received line (where the stars are anything). When it finds a match, it gives the message a SPAM score of -50 (ensuring it is not counted as SPAM). |
| Optional Score Increases | In this section, I turn up the value of several of the rules. The default score for a spam that turns up in the DCC database is only 2.756 when we're using Bayes and network checks. This seemed a little low for me, so I upped it to 4 points. If you wanted every message listed in the DCC database to be tagged as SPAM, you'd set this to 6.3 points. You can check the default scores for everything in the file /usr/local/share/spamassassin/50_scores.cf. You may see 4 different scores listed next to some rules. The file has different scores for whether or not you are using Bayes and network checks. When there is only 1 score, that score applies all the time, otherwise the 4th score is for bayes and network checks like we are using. |
Unfortunately I couldn't get Razor working correctly when not running as chroot. For some reason it had a hard time figuring out what its razorhome directory was (even though it was explicitly set in the config file). Fortunately it works great in chroot mode. If you are not going to run amavisd-new in chroot mode, you may want to just disable the razor checks in your /etc/mail/spamassassin/local.cf file.
Download latest v2 razor-agents Untar and run
# cd /root
# ftp http://unc.dl.sourceforge.net/sourceforge/razor/razor-agents-2.40.tar.gz
# tar -zxvf razor-agents-2.40.tar.gz
# cd razor-agents-2.40
# perl
Makefile.PL
# make
# make
test
# make
install
Create the default configuration files in /etc/razor
#
razor-client
# razor-admin
-create
Register yourself with the razor network. Substitute your exchange email address
for the address listed.
# razor-admin
-register -user postmaster@domain.com
Copy the razor config files to the chroot directory
# cp -r /root/.razor /var/amavisd
# chown -R amavisd.amavisd
/var/amavisd/.razor
Unfortunately you have to install a patch to Razor to get it to work properly
when called by SpamAssassin. Unless you have logging set to the highest levels,
Razor will actually appear to be working, but will never tag any messages. Fortunately
applying the patch is easy enough.
# cd /usr/local/libdata/perl5/site_perl/i386-openbsd/Razor2/Client
# ftp http://www.ijs.si/software/amavisd/Razor2.patch-quinlan
# patch < Razor2.patch-quinlan
The patch program will automatically apply the patch and make a backup of the
original file. Otherwise, that's it, Razor is now ready to go.
You can enable or disable Razor by editing your etc/mail/spamassassin/local.cf
file: use_razor2 1
One last note, razor has its own logfile in /var/amavisd/.razor called
razor-agent.log. Unchecked, this file could potentially grow to fill your entire
/var volume. Therefore, once you are sure that razor is working properly, you
will want to edit the razor config to turn off logging. Unfortunately you can't
use newsyslog to rotate this logfile without having to stop and restart amavisd.
Edit /var/amavisd/.razor/razor-agent.conf and change:
debuglevel = 5
to
debuglevel = 0
then restart amavisd.
Download and extract the latest DCC (http://www.dcc-servers.net/dcc/)
#
cd /root
# ftp http://www.dcc-servers.net/dcc/source/dcc-dccd.tar.Z
# tar -zxvf dcc-dccd.tar.Z
# cd dcc-dccd-1.2.36
# ./configure
# make
# make
install
Make sure udp port 6277 is allowed out from this computer on your firewall.
# /usr/local/bin/cdcc
'info'
If everything is working, you should see a bunch of lines like:
dcc.rhyolite.com, - RTT+0 ms anon
# 153.19.44.233,- coral.ely.pg.gda.pl WEiAPG server-ID 1072
# 100% of 3 requests ok 1687.64+0 ms RTT 113 ms queue wait
# 192.188.61.3,- calcite.rhyolite.com Rhyolite server-ID 101
# 100% of 2 requests ok 755.52+0 ms RTT 50 ms queue wait
Since amavisd-new will be running in chroot mode, we need to copy DCC
and all the files it needs to the chroot directory. The way DCC is called by
SpamAssassin requires /bin/sh in order to work properly. You should be aware
that this it reduces the security of the system. Without it, you'll receive
this error when running amavisd debug: DCC -> check failed: no response
# mkdir -p /var/amavisd/var
/var/amavisd/usr/bin /var/amavisd/usr/libexec /var/amavisd/var/dcc
# mkdir -p /var/amavisd/usr/lib /var/amavisd/bin
# cp -r /var/dcc /var/amavisd/var/
# cp /usr/local/bin/dccproc
/var/amavisd/usr/bin
# cp /usr/libexec/ld.so /var/amavisd/usr/libexec
# chown -R amavisd:amavisd
/var/amavisd/var/dcc
# cp /bin/sh /var/amavisd/bin/
DCC uses several OpenBSD libraries, which we need to copy to the chroot directory. Since the version numbers on these libraries seem to change regularly, rather than just telling you which files to copy, you can run the following command:
# ldd /usr/local/bin/dccproc
This will output something like this:
/usr/local/bin/dccproc:
Start End Type Ref Name
00000000 00000000 exe 1 /usr/local/bin/dccproc
00008000 2000f000 rlib 1 /usr/lib/libm.so.1.0
00019000 20063000 rlib 1 /usr/lib/libc.so.30.0
00002000 00002000 rtld 1 /usr/libexec/ld.so
Just copy the listed files (except dccproc) to their chroot directory under
/var/amavisd. So for example, you would copy /usr/lib/libm.so.1.0
to /var/amavisd/usr/lib/libm.so.1.0.
You can enable or disable DCC by editing your etc/mail/spamassassin/local.cf file: use_dcc 1
Installing and Configuring Amavisd-new
use strict; $MYHOME = '/var/amavisd'; $QUARANTINEDIR = "$MYHOME/quarantine"; $forward_method = 'smtp:127.0.0.1:10025';
@bypass_virus_checks_acl
= qw( . ); $DO_SYSLOG = 1; # (1 = syslog,
0 = logfile) $hdrfrom_notify_sender =
'SpamAssassin <helpdesk@domain.com>'; $final_spam_destiny = D_PASS; # Set to D_BOUNCE to block/notify, D_PASS to pass through read_hash(\%whitelist_sender,
'/var/amavisd/whitelist'); $path = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/usr/bin:/bin'; $file = 'file'; $sa_tag_level_deflt = 3.0;
# controls adding the X-Spam-Status and X-Spam-Level headers,
1; # insure a defined return |
| Explanation of Amavisd-new configuration settings: | |
| $MYHOME, $mydomain | These are really just variables used in this config file so we don't have to type the same things over and over again. MYHOME is our default work directory, mydomain is the name of our domain. |
$daemon_user, |
This is the user and group that amavisd and all its helper programs will run under for security reasons. |
| $daemon_chroot_dir | This option tells amavisd to run in chroot mode. Chroot is a security trick that traps the program in a certain directory, in our case $MYHOME. By turning this on, amavisd will not be able to access anything on the computer other than what is in the /var/amavisd directory. |
| $QUARANTINEDIR |
This is the location where amavisd would place quarantine files if you were using it for virus scanning. Even though we aren't virus scanning, it is still needed. |
| $TEMPBASE | When amavisd does its thing, it creates a bunch of temp files. Normally it would just store them in $MYHOME. By storing them in $MYHOME/tmp, we can easily cleanup the temp directory whenever we need to by stopping amavisd and deleting everything in $MYHOME/tmp/* |
| $ENV{TMPDIR}, $helpers_home |
I found that both of these setting help SpamAssassin, Razor and DCC work better by explicitly telling these helper programs their home and tmp directories. |
| $forward_method, $notify_method, $inet_socket_port, $inet_socket_bind, $inet_acl |
These settings set up the communications between amavisd and postfix. Basically the two programs communicate using different ports. Postfix sends email it wants to filter to amavisd on port 10024. Amavis processes the message and returns it to postfix on port 10025. The $inet_acl setting makes sure that it accepts only packets from the local computer. |
| @bypass_virus_checks_acl | Since we won't be using the anti-virus features of amavisd, this line is used to turn them off. |
| @local_domains_acl | This setting is used to determine if a message is incoming or outgoing. |
| $DO_SYSLOG, $LOGFILE, $log_level |
These settings describe how amavisd should do logging for debugging.
The $log_level can be set from 0-5 with 5 producing the most logging
output. Even though we aren't using it, the $LOGFILE setting is required.
Also don't be surprised to see an empty file with this name in the $MYHOME
directory. You should also note that when running amavisd in debug mode,
logging doesn't occur to the syslog. Note #1: the main reason we don't want to log to file is because when chrooted, amavisd can't be restarted with a HUP command. Therefore to rotate the logfile with newsyslog, you'd need to stop and restart amavisd. It's easier just to log to syslog. Note #2: if you want to have amavisd log to /var/log/messages instead of /var/log/maillog, you can add a config line here: $SYSLOG_LEVEL = 'user.info'; |
| $hdrfrom_notify_sender | When we send email messages to the spammers telling them we are rejecting their email, and giving them directions for how to get on our whitelist, this is who the message appears to be from. |
| $notify_spam_sender_templ | This is where we setup our custom response file. Basically when we set the $final_spam_destiny to D_BOUNCE, all spam will be rejected and the spammer will get this notice. The notice is really meant for our clients that may have been falsely accsed of sending spam. It tells them why their message was identified as spam and gives them a phone number they can call to get on our white list. (see below for the contents of my file) |
| $final_spam_destiny | You should set this to either D_BOUNCE or D_PASS. When set to D_PASS,
the spam will be delivered to your users. They can then setup inbox
assistant rules looking for the X-Spam-Flag header to move the messages
to another folder, or delete the messages. The D_BOUNCE setting will
reject the spam message all together so it never reaches your exchange
server. Instead, the sender of the spam message will receive a nice
email from this computer. The email is really meant for our clients
that may have been falsely accsed of sending spam. It tells them why
their message was identified as spam and gives them a phone number they
can call to get on our white list. At my company, we used the D_PASS method for the first 60 days after turning on SpamAssasssin to make sure everything was working right (auto-whitelisting had been done, and the bayesian db had been built). Then we switched over to D_BOUNCE mode. Amavisd also supports D_DISCARD and D_REJECT, which are described in their documentation. |
| read_hash(\%whitelist_sender, read_hash(\%blacklist_sender, read_hash(\%spam_lovers |
These settings point to 3 files that identify our whitelist, blacklist
and spam lovers. Each file should have 1 email address (or part of an
email address) per line. The whitelist identifies senders that should
always be passed through even if they are identified as spam. The blacklist
identifies senders that should always be marked as spam. The spam_lovers
identifies our users that want to opt-out of our system and receive
all their spam. Note that these files are read only on startup of amavisd, so if you edit them be sure to stop and restart amavisd. Also, not shown here is the ability of amavisd to support per-recipient whitelists and blacklists and MySQL based whitelists and blacklists. |
| $MAXLEVELS, $MAXFILES, $MIN_EXPAN... | This whole section is part of the anti-mail bomb measures of the anti-virus software. I thought it should be kept in there just in case. |
| $path | Tells amavisd where to look for programs it needs. |
| $banned_filename_re | Currently needed even though we aren't using it to block illegal attachment types. This is a workaround to fix a bug where SA inserts the ***SPAM*** into the subject twice for some messages. This should be fixed in a future release of amavisd-new. |
| $file, $arc, $gzip, $bzip2, etc. etc. | These are the names of the programs that amavisd would normally use to unzip files so it could scan them for viruses. Basically we make them all available to amavisd to avoid any error messages, but they shouldn't ever be used since we turned of the anti-virus code. |
| $sa_local_tests_only | Set this to 1 to disable DCC and Razor for debugging. The default for this is false, so I initially didn't include it in my config file, but I have been told that leaving it out causes problems. |
| $sa_auto_whitelist | Great feature of spamassassin that automatically whitelists people that send you lots of non-spam mail. |
| $sa_mail_body_size_limit | Since few spammers send large attachments, we don't even bother checking messages over a certain size (in this case 64KB). |
| $sa_tag_level_deflt, $sa_tag2_level_deflt, $sa_kill_level_deflt |
As you know, SpamAssassin assigns each email a positive or negative score to indicate its "spamminess" (yes I know its not a word). These settings tell SA when to take anti-spam measures. At the tag level, the message's headers are modified with the spam score. At the tag2 level, in addition to the actions from the tag level, it adds a header indicating that the message is spam, and it modifies the subject. At the kill level, the system will take action based on the $final_spam_destiny setting. In most cases, you want tag2 and kill to happen to any message that is spam, so they are usually equal. |
| $sa_spam_subject | This text is added to the front of any spam message that receives a high enough score. |
| $sa_debug | This setting turns on spamassassin debugging. Comment it out once everything is running smoothly for better performance. |
From: SpamAssassin <helpdesk@domain.com> Subject: **Message you sent blocked by our SPAM filter** [? %m |#|In-Reply-To: %m] Message-ID: <SS%n@%h> Your message to: %R Subject: %j Our company uses a set of email filters to help block the delivery of unsolicited commercial email, otherwise known as SPAM. For more information on SPAM, please visit http://spam.abuse.net. If you believe that you have received this message in error, please accept our sincere apologies. We ask that you please reply to this email message. When we receive your reply, we will add your email address to our whitelist of approved senders so that in the future we can avoid making this mistake again. Please note that this is a manual process and is only done during business hours. The report below will help you determine why your message was flagged as SPAM. If you continue to have problems, please contact our Helpdesk at 800-555-1212. Thank you very much, Postmaster
|