11.10.2017

How to set up Fetchmail, Maildrop and Dovecot as a spam filtering IMAP “proxy”


Why?

OVH is my web hoster and domain name registrar, it also provides me with an email service for my domain names, e.g.: xxxx@mrhide.eu.

It comes with an SSL encrypted POP3 and IMAP endpoint, anti-spam, and a webmail GUI (Roundcube).
Their anti-spam feature adds a header and prepend the subject with [SPAM], there is one option to automatically delete junk email, or forward it to another address, but there is no option to put it in a junk or spam folder.
I could have made it delete all junk emails, but there are many false positive!

They have a paid for Exchange (Microsoft) service that comes with many more features.
Because they don't want their free offer to compete with the commercial one, they only provide a crippled free service.

What happens if you ask them for server-side filtering? They either tell you to upgrade your email plan, or to set up Google Apps to benefit from Gmail's server-side filtering feature.

I can use client-side filtering, it works well in thunderbird, but the android email client misses that feature ...

I came up with a not so simple, not so elegant solution, that requires a dedicated server (that I already have).

Note: this article is a simple workaround, there is a more professional approach https://www.rosehosting.com/blog/how-to-set-up-server-side-email-filtering-with-dovecot-sieve-and-roundcube-on-a-centos-6-vps/.

Requirements

  1. Fetch emails from the IMAP service at OVH.
  2. Do not remove emails from the original IMAP service (If my server crashes, I don't want to lose emails).
  3. Put spam in the junk folder.
  4. Create a new IMAP service to serve the filtered mail box.

Fetching and filtering emails

Install Fetchmail, Maildrop and Dovecot, these are widely used programs and should be available through your Linux distribution's package manager.
Fetchmail will act as a MTA (Mail Transfer Agent), it will fetch mails from my hoster, and send them to Maildrop, our MDA (Mail Delivery Agent) (or LDA (Local Delivery Agent)). Maildrop can filter emails based on a language that applies on the source file of an email (its headers and content) and put it in a mailbox. Then Dovecot will serve emails from that mailbox.

All the following shell commands must be run as root.

First we need to create a user that will contain dotfiles to configure software and our mailbox:

groupadd -g 5000 emailproxy
useradd -m -d /srv/emailproxy -s /bin/false -u 5000 -g emailproxy emailproxy

Second we create a mailbox using the maildir++ format:

maildirmake /srv/emailproxy/.maildir

We also create a Junk folder in our mailbox:

maildirmake -f Junk /srv/emailproxy/.maildir

Created file structure should look like this:

tree -a /srv/emailproxy/.maildir
/srv/emailproxy/.maildir
├── cur
├── .Junk
│   ├── cur
│   ├── maildirfolder
│   ├── new
│   └── tmp
├── new
└── tmp
‌
7 directories, 1 file

Create a .mailfilter filter for Maildrop (this file will be parsed after /etc/maildroprc):

cat > /srv/emailproxy/.mailfilter << EOF
if (/^X-Ovh-Message-Type: SPAM$/)
    to $DEFAULT/.Junk/
EOF

This file tells Maildrop to put emails in the Junk folder if it matches a pattern. Here, the header added by OVH's anti-spam service. Of course, you can define more rules to sort your emails here, eg: put email from mailing lists in a special folder.

Create a .fetchmailrc file to configure Fetchmail:

cat > /srv/emailproxy/.fetchmailrc << EOF
poll ssl0.ovh.net protocol imap username "××××××××" with ssl with password "××××××××" mda "/usr/bin/maildrop" keep
EOF

The syntax is verbose and explicit, it's just like an order you could have told to somebody.
Fetchmail when started will read this file, poll the given IMAP service, fetch all new emails, pass them to Maildrop and mark them as read, it will not delete emails (keep).

Finally, fix ownership and permissions:

chown -R emailproxy:emailproxy /srv/emailproxy/
chmod -R 2770 /srv/emailproxy

Invoking fetchmail as user emailproxy should populate the maildir mailbox.

Serving emails

First, I don't want anyone but me to be able to read my emails, so I decided to use SSL here. Hopefully dovecot has a script to ease the generation of an RSA key and an SSL certificate, it should be in its documentation: /usr/share/doc/dovecot/mkcert.sh using the configuration file /usr/share/doc/dovecot/dovecot-openssl.cnf (that script is short and very easy to understand, and can be customized using environment variables).

Invoking mkcert.sh will create a key in /etc/ssl/private/dovecot.pem and generate an SLL certificate in /etc/ssl/certs/dovecot.pem.

First I'll edit /etc/dovecot/dovecot.conf to override one parameter:

# Only serve using IMAP protocol
protocols = imap

Then I'll tell it where to look for inboxes, overriding only one parameter in file /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/.maildir

Then I'll enable SSL, in file /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/ssl/certs/server.pem
ssl_key = </etc/ssl/private/server.key

Finally, I'll configure dovecot to use a passwd user/password database in file /etc/dovecot/conf.d/auth-passwdfile.conf.ext:

passdb {
  driver = passwd-file
  args = scheme=CRYPT username_format=%u /etc/dovecot/users
}
‌
userdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users
}

And the corresponding /etc/dovecot/users file:

emailproxy:××××××××:5000:5000::/srv/emailproxy:

The syntax here is very simple, it's the same as /etc/passwd, because the purpose of this driver is tu use that very file. For more information, see dovecot's wiki page.

Systemd setup

Because systemd is sacred, systemd is great! ... nah I really like systemd, it is a nice tool, and it's the default init daemon of my favourite Linux distribution.

Autostart fetchmail daemon for user emailproxy:

cat > /etc/systemd/system/fetchmail-proxy.service << EOF
[Unit]
Description=A remote-mail retrieval utility
After=network.target
‌
[Service]
User=emailproxy
ExecStart=/usr/bin/fetchmail -d 60
RestartSec=1
‌
[Install]
WantedBy=multi-user.target
EOF
systemctl enable fetchmail-proxy
systemctl start fetchmail-proxy

Autostart dovecot: my distro has a correct systemd unit for dovecot, so I will just enable and start it:

systemctl enable dovecot
systemctl start dovecot

Client setup

I will use OVH's SMTP server for outgoing emails and my newly setup IMAP server for incoming emails, I found no reason to put a proxy in front of a SMTP that I don't control.

Use the credentials defined in the /etc/dovecot/users file, the IP or domain name of the machine hosting our proxy, port is 993 (SSL).
Usually you'll have to tell your client to trust the self signed certificate generated by the mkcert.sh script.

And .... we're done, I don't know if it's the best setup to achieve that simple goal, but it works well for me!