Running your own mail server teaches you exactly how email works, and gives you full control. The classic open-source combination is Postfix (sends and receives mail via SMTP) and Dovecot (stores mailboxes, serves IMAP, and authenticates users for Postfix). This guide builds a working, secure single-domain server on Ubuntu. For a ready-made bundle instead, see mailcow or Mail-in-a-Box.

Prerequisites

  • A VPS where outbound port 25 is allowed, and reverse DNS can be set. On VPS.DO, both are available on USA VPS plans. See email VPS.
  • Hostname mail.example.com, an A record to the server IP, and PTR set to mail.example.com. See PTR records.
  • MX record: example.com. MX 10 mail.example.com. See MX records.
sudo hostnamectl set-hostname mail.example.com

1. TLS certificate

sudo apt install certbot
sudo certbot certonly --standalone -d mail.example.com

2. Install the packages

sudo apt install postfix dovecot-core dovecot-imapd dovecot-lmtpd

In the Postfix dialog, choose Internet Site, with the system mail name example.com.

3. Mail storage and users

This setup uses system users for simplicity: each mailbox is a Linux account, and mail is stored in ~/Maildir. Virtual users (from a database or a passwd-file) are better for many domains. The configuration differs slightly.

sudo adduser --disabled-login alice     # set a password for mail login with: sudo passwd alice

4. Dovecot configuration

/etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

/etc/dovecot/conf.d/10-auth.conf:

disable_plaintext_auth = yes
auth_mechanisms = plain login

/etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem
ssl_min_protocol = TLSv1.2

/etc/dovecot/conf.d/10-master.conf: expose authentication and LMTP to Postfix:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}
sudo systemctl restart dovecot

5. Postfix configuration

/etc/postfix/main.cf (key settings):

myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
mydestination = $myhostname, $mydomain, localhost
inet_interfaces = all
inet_protocols = ipv4

# deliver to Dovecot
mailbox_transport = lmtp:unix:private/dovecot-lmtp

# TLS
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file  = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtp_tls_security_level = may
smtpd_tls_auth_only = yes

# SASL auth via Dovecot
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

# no open relay
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

inet_protocols = ipv4 avoids IPv6 deliverability problems if your IPv6 address has no PTR record.

In /etc/postfix/master.cf, enable the submission (587) and submissions (465) services with mandatory authentication. See SMTP security for the full blocks.

sudo postfix check && sudo systemctl restart postfix

6. Firewall

sudo ufw allow 25,465,587,993/tcp

7. DKIM, SPF and DMARC

Sign outgoing mail with OpenDKIM (or rspamd), and publish SPF and DMARC records:

example.com.         TXT  "v=spf1 mx -all"
_dmarc.example.com.  TXT  "v=DMARC1; p=none; rua=mailto:[email protected]"

Full instructions: SPF, DKIM and DMARC.

8. Spam filtering

At minimum, add Postfix restrictions (reject_unknown_client_hostname, RBL checks such as reject_rbl_client zen.spamhaus.org, through your own resolver). For proper filtering, add rspamd, which also handles DKIM signing and greylisting. Protect logins with Fail2Ban jails for postfix-sasl and dovecot. See Fail2Ban.

9. Test

swaks --to [email protected] --from [email protected] --server mail.example.com:587 --tls --auth-user alice --auth-password '…'
sudo tail -f /var/log/mail.log

Then check the received message headers (SPF, DKIM and DMARC pass), and score the setup at mail-tester.com. Connect a mail client: IMAP mail.example.com:993 (SSL), SMTP mail.example.com:587 (STARTTLS). See email deliverability.

Ongoing care

  • Reload Postfix and Dovecot after certificate renewals (a Certbot deploy hook).
  • Watch the queue (mailq), the logs and the blocklists. See IP blacklists.
  • Back up /home/*/Maildir and /etc/postfix, /etc/dovecot. See backup strategy.
  • Add webmail with Roundcube.

Quick reference

Component Role / port
Postfix SMTP: 25 (inbound), 587/465 (submission)
Dovecot IMAP 993, SASL auth, LMTP delivery
Config /etc/postfix/main.cf, master.cf, /etc/dovecot/conf.d/
Check postfix check, doveconf -n
Logs /var/log/mail.log

Related: SMTP ports and transactional email services.

  • Email
  • Postfix
  • Linux