Postfix Built-in Content Inspection

Built-in content inspection introduction

Postfix supports a built-in filter mechanism that examines message header and message body content, one line at a time, before it is stored in the Postfix queue. The filter is usually implemented with POSIX or PCRE regular expressions, as described in the header_checks(5) manual page.

The original purpose of the built-in filter is to stop an outbreak of specific email worms or viruses, and it does this job well. The filter has also helped to block bounced junk email, bounced email from worms or viruses, and notifications from virus detection systems. Information about this secondary application is given in the BACKSCATTER_README document.

Because the built-in filter is optimized for stopping specific worms and virus outbreaks, it has limitations that make it NOT suitable for general junk email and virus detection. For that, you should use one of the external content inspection methods that are described in the FILTER_README, SMTPD_PROXY_README and MILTER_README documents.

The following diagram gives an over-all picture of how Postfix built-in content inspection works:

Postmaster
notifications
|
v
Network or
local users 		-> Built-in
filter 		-> Postfix
queue 		-> Delivery
agents 		-> Network or
local mailbox
^
| 		|
v
Undeliverable mail
Forwarded mail

The picture makes clear that the filter works while Postfix is receiving new mail. This means that Postfix can reject mail from the network without having to return undeliverable mail to the originator address (which is often spoofed anyway). However, this ability comes at a price: if mail inspection takes too much time, then the remote client will time out, and the client may send the same message repeatedly.

Topics covered by this document:

What mail is subjected to header/body checks

Postfix header/body checks are implemented by the cleanup(8) server before it injects mail into the incoming queue. The diagram below zooms in on the cleanup(8) server, and shows that this server handles mail from many different sources. In order to keep the diagram readable, the sources of postmaster notifications are not shown, because they can be produced by many Postfix daemon processes.

bounce(8)
(undeliverable)
smtpd(8)
(network) 		\ |
v
qmqpd(8)
(network) 		-\
-/ 		cleanup(8) -> incoming
queue
pickup(8)
(local) 		/ ^
|
local(8)
(forwarded)

For efficiency reasons, only mail that enters from outside of Postfix is inspected with header/body checks. It would be inefficient to filter already filtered mail again, and it would be undesirable to block postmaster notifications. The table below summarizes what mail is and is not subject to header/body checks.

Message type Source Header/body checks?
Undeliverable mail bounce(8) No
Network mail smtpd(8) Configurable
Network mail qmqpd(8) Configurable
Local submission pickup(8) Configurable
Local forwarding local(8) No
Postmaster notice many No

How does Postfix decide what mail needs to be filtered? It would be clumsy to make the decision in the cleanup(8) server, as this program receives mail from so many different sources. Instead, header/body checks are requested by the source. Examples of how to turn off header/body checks for mail received with smtpd(8), qmqpd(8) or pickup(8) are given below under "Configuring header/body checks for mail from outside users only" and "Configuring header/body checks for mail to some domains only".

Limitations of Postfix header/body checks

Preventing daily mail status reports from being blocked

The following is quoted from Jim Seymour's Pflogsumm FAQ at http://jimsun.linxnet.com/downloads/pflogsumm-faq.txt. Pflogsumm is a program that analyzes Postfix logs, including the logging from rejected mail. If these logs contain text that was rejected by Postfix body_checks patterns, then the logging is also likely to be rejected by those same body_checks patterns. This problem does not exist with header_checks patterns, because those are not applied to the text that is part of the mail status report.

You configure Postfix to do body checks, Postfix does its thing, Pflogsumm reports it and Postfix catches the same string in the Pflogsumm report. There are several solutions to this.

Wolfgang Zeikat contributed this: 

#!/usr/bin/perl
use MIME::Lite;

### Create a new message:
$msg = MIME::Lite->new(
    From     => '[email protected]',
    To       => '[email protected]',
    # Cc     => '[email protected], [email protected]',
    Subject  => 'pflogsumm',
    Date     => `date`,
    Type     => 'text/plain',
    Encoding => 'base64',
    Path     => '/tmp/pflogg',
);

$msg->send;

Where "/tmp/pflogg" is the output of Pflogsumm. This puts Pflogsumm's output in a base64 MIME attachment.

Note by Wietse: if you run this on a machine that is accessible by untrusted users, it is safer to store the Pflogsumm report in a directory that is not world writable.

In a follow-up to a thread in the postfix-users mailing list, Ralf Hildebrandt noted:

"mpack does the same thing."

And it does. Which tool one should use is a matter of preference.

Other solutions involve additional body_checks rules that make exceptions for daily mail status reports, but this is not recommended. Such rules slow down all mail and complicate Postfix maintenance.

Configuring header/body checks for mail from outside users only

The following information applies to Postfix 2.1 and later. Earlier Postfix versions do not support the receive_override_options feature.

The easiest approach is to configure ONE Postfix instance with multiple SMTP server IP addresses in master.cf:

Configuring header/body checks for mail to some domains only

The following information applies to Postfix 2.1. Earlier Postfix versions do not support the receive_override_options feature.

If you are MX service provider and want to apply disable head/body checks for some domains, you can configure ONE Postfix instance with multiple SMTP server IP addresses in master.cf. Each address provides a different service. 

/etc/postfix.master.cf:
    # =================================================================
    # service     type  private unpriv  chroot  wakeup  maxproc command
    #                   (yes)   (yes)   (yes)   (never) (100)
    # =================================================================
    # SMTP service for domains with header/body checks turned on.
    1.2.3.4:smtp  inet  n       -       n       -       -       smtpd

    # SMTP service for domains with header/body checks turned off.
    1.2.3.5:smtp  inet  n       -       n       -       -       smtpd
        -o receive_override_options=no_header_body_checks

Once this is set up you can configure MX records in the DNS that route each domain to the proper SMTP server instance.