David McNett :: Postfix Relay Recipient Maps

Rough Draft Documentation and Code by David McNett

Setting up automatic relay_recipient_maps in postfix

SMTP relies on the concept of MX records to provide redundancy for incoming emails. A domain lists one or more MX records in DNS which point to a ranked listing of hosts which are configured to receive emails on behalf of that domain. One common configuration is to have a single "primary" mail host which actually does the mail delivery to end users. This primary mail host is supported by "backup mx hosts" which serve only to receive and queue mail from senders in the event that the primary mail host is unavailable. When the primary mail host returns, these backup mx hosts then forward their stored mail to the primary without ever having actuall processed or understood the mails.

This system is easy to establish, simple to maintain and does a great job at making sure that nobody's mail gets dropped on the floor and lost in the event of a system or network failure. It is, however, subject to one significant liability in today's age of spam and email worms. The problem stems from spammer's mailblasting software and internet worms not using the MX records as they were intended. In practice we see that spammers will indiscriminately send mail to any one of a domain's MX hosts seemingly at random instead of trying the servers in the specified order.

The downside to this is that backup mx hosts are rarely smart enough to be able to discard invalid recipient addresses in the same way that a primary mail server can easily do. The primary mail server can instantly know that nobodyhome@slacker.com is not a valid address and can accordingly reject the mail as spam without the expense of queuing and attempted delivery. A backup MX does not have this luxury. As a result, backup mx's cause a lot of grief and work because they act as a backdoor for spammers to bypass the effective rejection of invalid recipients. While none of the invalidly-addressed mail will bother a real user (by virtue of their invalid recipient addresses), they do clog up both systems and increase the burden on spam filters and virus scanners. A far better approach is to provide a backup MX host with the information it needs to discard invalid recipients without bothering the primary MX server.

The Postfix mailer provides a facility for doing exactly this, but there's not much more than some terse and literal documentation along with a bit of hand-waving about how an administrator might actually accomplish this in a real-world environment. The reality is, any implementation is going to vary wildly based on the site platform, needs, and paranoia. That said, in an effort to fill the void left by the spartan Postfix docs, this page will document how one site implemented relay_recpient_maps and will hopefully provide you with useful tools and ideas which can be adapted to your specific needs.

First we need a list

At its core, the foundation of using relay_recipient_maps is the creation of a master list of valid addresses for a site. To this end, I wrote a perl script which culls the contents of /etc/passwd and /etc/aliases into a properly-formatted hash list for postfix to use. This is only useful on UNIX hosts which have only actual unix account users. If you're using any sort of virtual users table, this script will be insufficient for you. The script is here:

relay_recipients.pl

This script, when run, will export a hash file suitable for use on the backup MX's installation of postfix.

NOTE: when postfix is evaluating addresses for relay_recipient_maps, it does not include any address expansions in the evaluation. This means that if you have listed, for example, nugget@slacker.com as a valid recipient, then the backup MX will happily relay mail to any expanded form of that address. nugget+foo@slacker.com or nugget+bar@slacker.com will both be relayed even if there is no corresponding .forward-foo or .forward-bar file set up for nugget. While this is is not ideal, it's well worth it for the simplicity it allows in this syncing process. As you might imagine, trying to keep an up-to-the-minute list of valid expansion addresses is beyond impractical.

Getting that list where it is needed

The second and equally important part of this process is finding a way to get this map file onto each backup mx server so it can be used. The relay_recipients.pl script also contains this capability. But first, you'll need to create a passphraseless ssh key which will be used to conduct the file transfer. Here's how (just hit enter for the passphrase):

      user@primarymx$ ssh-keygen -t dsa -f relay_identity
      Generating public/private dsa key pair.
      Enter passphrase (empty for no passphrase): 
      Enter same passphrase again: 
      Your identification has been saved in relay_identity.
      Your public key has been saved in relay_identity.pub.
      The key fingerprint is:
      28:8a:49:7c:b6:1f:99:e7:e8:e3:fa:03:1c:0a:67:9a user@primarymx

This will have generated two files: relay_identity and relay_identity.pub. Place these in your postfix config directory (mine is /usr/local/etc/postfix/) and make sure that the $scp_flags variable in relay_recipients.pl has the proper path. Neither file should be readable by users, only by the user which will be running the relay_recipients.pl script.

Next, edit the @scp_targets array in the script so that it contains one or more entries. There should be one array element for each backup mx which will be receiving the published map. Each element is in the form of username@hostname:directory and "." can be used to represent the user's home directory.

Finally, log in to each backup max host as the user specified in the array and create a .ssh directory and an authorized_keys2 file. The authorized_keys2 file should contain the contents of the relay_identity.pub file you have just created. Permissions are critical on this directory and this file:

      user@backupmx$ cd $HOME
      user@backupmx$ mkdir .ssh
      user@backupmx$ chmod 700 .ssh
      user@backupmx$ cd .ssh
      user@backupmx$ touch authorized_keys2
      user@backupmx$ chmod 600 authorized_keys2

Using your favorite editor, edit authorized_keys2 and paste in the contents of relay_identity.pub. The whole bit of data should be all on one line with no linebreaks. If you're a pico user (ewww), make sure to turn off wordwrap by using "pico -w authorized_keys2". For additional security, you can add some additional restrictions to the entry. Here's what mine looks like (displayed wrapped to 80 cols):

      from="primarymx_hostname",no-pty,no-port-forwarding ssh-dss AAAAB3NzaC1kc3MAAACB
      MdB/X4vZGuLLj1UqfJYuoQnsB7hCmnKXdXo5NiHrFFar9YPFsBy48obPU0ENkaBBu2GzaQIMetx88gAP
      hP1KYxeC2AkOsxxmBMuQx4Kczw612IxaESr9dmW+0ByAQp4rR2doekYTTtaFPWz8+jJcrrjzh7CJ08mG
      V+hDe0ny1qVAAAAFQCUI43ueY7DVoiQf1O0jvfHCssTWwAAAIEAqBhpzZeUWaa0yleUwgduDAISqW2To
      u9rcCCEQxbI2V6TkTXTdaewYaTJxNex+rALnTjq5zwcGlXtLxQOnGzWwcjeITOEaDD10TGJ4uaYjg0Vb
      AZM2LedKpVQHoLdQSUIkTMxT6N/B3rZHFHtKKbJZ+RVXTPr/ftakCzHSxK+6D4AAACAU7z1oGuT3kQYA
      mnA+qIrwih4iRo+xyHeoPshd/NQ37FPq8cYTkhdlmA4shvP9JJWyrlJO/BCRqjW2R/WCucUqff4MIqbg
      j/1JP4uIGAnMBNByA8Xbdnu0u2e3NCaGdrFoHs5TtmeBbxaiOMvOm8GjkNyQmnAkf1oz8L7UMArJeE= 
      postfix relay_recipients key

That last bit (still all on the same line) will probably be your email address. It's just a comment and can say whatever you wish.

You should now be able to run the relay_recipients.pl script on your primary mx host and it will compile a list of valid recipients and if the list has changed it will scp it (as raw hash and db file) to each secondary mx target that you have defined.

What's a secondary got to do

The last step is to tell the secondary where to find the recipient map. This is done using the relay_recipients_map facility in main.cf. Here's what one looks like:

       relay_recipient_maps = 
           hash:/usr/local/etc/postfix/relay_recipients
           hash:/home/nugget/relay_slacker

The relay_slacker file is the one that's being synced with our perl script. The other file is very important as well, though! It's a raw list containing wildcard entries for all the relay domains for which we're not tracking a map of valid recipients. THIS IS CRITICAL! If you've defined any recipient maps at all, then postfix will expect all domains relayed to be covered by the recipient maps. You must include wildcard records in this other file for domains that you're not syncing. Like this:

       @example.com        OK
       @bovine.net         OK

If you fail to do this, then postfix will reject all mail for the other relay domains.

Next, do a "postfix reload" and run some tests to make sure relaying is running ok.

Automate this puppy

Finally, throw the perl script into crontab on the primary mx. It should produce no stdout data, but if there is a problem it will spit out stderr, so don't /dev/null the output. I cron it in root's crontab to run every 10 minutes. It's fast and nothing actually gets pushed out unless there's a change to the configuration.

If a change is detected, the updated map file gets pushed out to each backup mx. Postfix running on the back mx hosts will notice that the timestamp has changed on the map file and will automatically reload the recipient map.

With this mechanism in place your backup mx hosts will no longer blindly accept invalid addresses which will avoid cluttering up the queues with junk and also avoid sending out useless bounces as well.

Feedback welcomed: nugget@slacker.com