eMail

Goals

• Understand the basic steps in the mail delivery system.

– Understand the role of the Mail User Agent (MUA)
– Understand the role of the Mail Transport Agent
(MTA)
• Understand the basic strategies for handling email.
• Understand the basic protocols used to deliver and
transport mail.
• Introduce the fundamentals of managing user accounts
(since mail is a service for users)

Module Code and Module Title Title of Slides

 Electronic Mail

E-mail is the ONLY universal mission-critical application

• Each person/group will have various mission-critical
applications
• But the only application that everyone depends on is e-mail
Keeping e-mail flowing is a required task for most system
administrators.

For the administrator this means:

– Choosing and configuring a mail transport agent
– Thinking through
• the site’s e-mail model,
• user agents, and
• hardware required to provide this critical service.
Module Code and Module Title Title of Slides
 Electronic Mail Overview

smtp smtp smtp

MTA 25 25 MTA
587 Relay MTAs (local)
MSA (remote)
(local)
MDA Message
MDA
(LDA) (LDA)
Spool
Send

MUA MAA
pine Get imap/pop

MUA – Mail User agent Protocols

MSA – Message Submission agent SMTP – Simple Mail Transport Protocol
MTA – Mail Transport agent IMAP – Internet Mail Access Protocol
MDA – Mail Delivery agent POP – Post Office Protocol
MAA – Mail Access agent

Module Code and Module Title Title of Slides

 Electronic Mail Overview –
Sending
smtp smtp
MTA 25
587 Relay MTAs
MUA Send MSA (initial)
pine (local)
MDA Message
(LDA)
Spool

• A user creates a mail message using a mail user agent (MUA)

– When done creating, they “send”
• The MUA contacts a local message submission agent (MSA)
– The MSA passes the message to a mail transport agent
(MTA) for remote delivery or a mail delivery agent (MDA)
for local delivery (also called a local delivery agent (LDA))

Module Code and Module Title Title of Slides

 RFC 2476 + RFC 4409: Message Submission
smtp smtp smtp
MSA 587 MTA 25 25 MTA
(local) (initial) Relay MTAs (destination)

Defines the difference between mail transfer and message submission:

• Submission is intended to be from client to initial server.
– SMTP protocol, port 587
• Transfer is intended to be server to server (relay)
– SMTP protocol, port 25
• MSA is optional, all of its functions can be done by a local MTA
• Allows administrators to run two distinct services configured optimally
for each purpose rather than a single MTA service that has to make
allowances for different types of use

Module Code and Module Title Title of Slides

 Message, Envelope, Headers
The MSA places the message in an “envelope” for delivery.
• There is a header on the message and another header on the
envelope.
– These contain addresses and other information about the
message.
• Message headers can be (and are) easily forged by user.
• Users typically do not see the envelope.
– Envelope headers are (normally) created, changed, and
deleted by MTAs
• Every message is assigned a unique ID by each MTA that
handles it.
– This allows tracing the message from end to end (if log files
are available).
Module Code and Module Title Title of Slides
 Electronic Mail Overview –Submission
smtp smtp
MSA port 587 should be MSA 587 MTA 25
limited to internal hosts (local) (initial)

• MSA converts mail to “canonical form”, for example by adding

@domain to mail submitted with a simple user name
• MSA does other address rewriting, for example changing
user@host.domain to Given.Family@domain (masquerading)
• MSA authenticates clients: able to require encryption and
SMTP authentication for port 587 sessions (Week 8)
• No need for (serious, automatic) spam filtering
– admninstrator can monitor logs for suspicious sending activity

MSA is optional – these tasks can be done by a local MTA

Module Code and Module Title Title of Slides
 Electronic Mail Overview – Transfer
smtp smtp
MTA 25 25 MTA
(initial) Relay MTAs (destination)

The MTA parses the “To:” address, and does a DNS query for a
MX record to find the remote host’s MTA.
– If contact succeeds, the MTA transfers the mail using SMTP
– If contact fails, the MTA retries for some finite period of time.
– Relay MTAs repeat this process until the message reaches
the destination MTA
• In order to control spam, ISPs are blocking outbound traffic
destined for TCP port 25 from all but a few recognised systems
• Thus if you want to use a mail server outside of the ISPs
network you need to use port 587 to communicate with it and
then have it send email to the world on your behalf.
Module Code and Module Title Title of Slides
 Electronic Mail Overview – SMTP
1. sending end introduces itself
2. sending end tells who mail is from
3. sending end tells who mail is for
If the destination user is valid /
relaying is permitted, open a
spool, and continue the
process. More on this in a
minute…
4. Sending end transfers data
5. Sending end closes connection
Module Code and Module Title Title of Slides
SMTP protocol
HELO – introduce yourself
AUTH – authenticate this user
MAIL FROM – who
RCPT TO – (message to)
VRFY – see if this user exists.
EXPN – expand this address
and tell me who it is
DATA – body of the message
DSN – delivery status notice
RSET – reset the connection
NOOP – do nothing
VERB – verbose mode
QUIT – close the connection
 DNS Necessities
• Canonical name for hosts
– lookup record - Name to IP
• MX records
– so MTAs can find the address to connect to
example.com IN MX 10 mailhost.example.com.
– MSAs don’t really need MX, initial MTA can be hard coded
• Mailserver reverse lookup record - IP to Name
– for validation purposes

Module Code and Module Title Title of Slides

 Relaying

• MX (Mail eXchanger) record

– priority allows multiple possibilities for getting mail
delivered
• MTA configuration
– Accept mail for certain destinations, then pass it along
• open relay = bad netizen
– likely to be used for spam

Module Code and Module Title Title of Slides

 Electronic Mail Overview –
Delivery
MTA MDA Message
(destination) (LDA)
Spool

• The destination MTA checks the “To:” address to see if there is a

system wide alias for the user.
– If an alias exists, use it as the local account name
• Then it checks to see if an account exists.
– If no account, reject the message.
– If an account exists, check to see if there is a .forward file in
the home directory.
 If .forward exists, read it and follow the instructions
 If not, pass the message to the MDA.
• MDA deposits Incoming messages into /var/mail/<account>

Module Code and Module Title Title of Slides

 Electronic Mail Overview –
Access
Message
MAA MUA
Spool imap or pop Get pine

• When the recipient invokes their MUA, it uses an access protocol

to check the local spool - informs the user if there is a message.
• Most common access methods are

 Post Office Protocol (pop3)

– Mail is always downloaded, a copy may be kept on the server
– Doesn’t support multiple simultaneous logins
 Internet Mail Access Protocol (imap)
– Mail is left on server unless user chooses otherwise
– Mail folders can be managed on the server
– Best for users who access their e-mail from more than one client.
Module Code and Module Title Title of Slides
 Electronic Mail Overview – Bulk
Delivery
Let’s say we collect mail for our local users from our ISP without an MTA

smtp
587 MTA smtp MTA MAA
(initial) 25 (isp) imap/pop
MSA
(local)
MDA Message
MDA
(LDA) fetchmail
Spool Get
Send

MUA MAA
pine Get imap/pop

fetchmail is a POP and IMAP

Our users access their mail on our
client used for bulk collection
system, rather than the ISPs system
of email from a server

Module Code and Module Title Title of Slides

 MTAs (briefly)
MTA is a generic term - usually one of these popular packages:
• Postfix [http://www.postfix.org/]
– Default on SUSE Linux and NetBSD.
• Exim [http://www.exim.org/]
Celebrate
– Default on Debian GNU/Linux. Diversity!
• Sendmail [http://www.sendmail.org/]
– Default on Slackware and FreeBSD; a Sun spinoff is used
on Solaris.
• Courier [http://www.courier-mta.org/]
– inspired by qmail, but intends to do things right
– qmail [http://cr.yp.to/qmail.html] is an obsolete and
unmaintained (but frequently mentioned) server

Module Code and Module Title Title of Slides

 Many different configurations are
possible
• Local delivery for users with an account on the system
• Local delivery for virtual users (no account on the system)
• Virtual hosting for domains, mail delivered to local user (virtual
or actual)
• Mail sent to a gateway then forwarded on to individual
mailservers – MUA talks to configured gateway MTA or MSA
• Local mail sent to local mailserver, external mail sent to
gateway – MUA talks to localhost MTA or MSA
• One mail server for all users, typically imap access
Weird and wonderful combinations will work!
Lots of decisions to make, Lots of documentation to read

Module Code and Module Title Title of Slides

 How complicated is it?
• postfix
 sets 530+ defaults
• sendmail:
 55+ features
 180+ defines, many with several distinct options
these are used to configure
18 classes


16 rulesets


15 macros


9 m4 diversions (subroutines)


• dovecot:
 About 150 defaults plus (2*7) database options

Module Code and Module Title Title of Slides

 TinyNet
Configuration You are actually accustomed to this
setup already from using Webmail,
gmail, Outlook, Thunderbird, etc.

Gateway MTA postfix accepts subnet

MUA and Internet mail, forwards to Mailserver
pine
MUA pine talks
SMTP port 25 to gateway MTA
and IMAP to Mailserver

Local MTA postfix accepts

all mail from gateway

Local MDA dovecot

delivers mail to valid users

Local MAA dovecot

provides imap access

Module Code and Module Title Title of Slides

 Configuration Decisions
Done already to save you the trouble

• Postfix or Dovecot for local delivery

– Postfix LDA has better spam filtering capabilities
– Dovecot LDA uses same authentication for LDA and IMAP
– Dovecot LDA creates mailboxes for new (known) users
• Use Dovecot LDA until LDAP is set up and stable

• System or Virtual users (or both)

– system users need “nologin” account on server (Later Today)
– local mail may be delivered to local system, not the mailserver (mailbox
confusion)
– easier to treat all users the same way
• All users are virtual users - only IMAP accounts

Module Code and Module Title Title of Slides

 Configuration Decisions
Done already to save you the trouble

• MSA or MTA (or both)

– MSA most valuable for heavy Internet mail traffic
– Sendmail is easier to configure as MSA
• Could use this rather than postfix on gateway
– MSA needs encryption and authentication (Week 8)
• Keep it simple, but keep the alternative in mind

• {Sendmail + UW_IMAP} or {Postfix + Dovecot}

– Sendmail + UW_IMAP is standard for the distribution (less work)
– Sendmail and UW_IMAP documentation is not great (more work)
– Postfix + Dovecot need to be found, tested with distribution (more work)
– Postfix and Dovecot have excellent documentation (less work)
• No choice but to do some work

Module Code and Module Title Title of Slides

 Some things we do not use ...

• procmail (default delivery agent on some distributions)

– reads a rule set that can filter mail messages in a variety of ways.
– there is a lot about procmail on the web, mostly old because it is meant
for sophisticated local users
• Spam filters in MTA / MDA / MUA
– spamassassin for example
– Real-Time Block Lists (rbl in postfix, dnsbl in sendmail)
• databases of spammers maintained in DNS
• these blacklists can be found by querying a search engine.
• Virus filters in MTA / MDA / MUA
– clamav for example
• POP access for remote mail
– imap is more standard nowadays
– web-based mail is always imap

Module Code and Module Title Title of Slides

 Web-based Mail
• A web-based mail client is a suite of CGI programs that act as
a mail client using a web browser as the user interface.
• requires obtaining and installing the needed CGI programs
and supporting programs and adding them to the web server
configuration
– For example, SquirrelMail requires Apache, PHP, and an
IMAP server, as well as the SquirrelMail software.
• Web-based mail can be used from any client platform that has
a web browser.
• If the web server offers encrypted connections, web-based
mail can make use of the encrypted connection to protect the
user ID, password, and message content. (Coming Soon)

Module Code and Module Title Title of Slides

 Problem Solving: Talking
SMTP
telnet hostname 25
HELO example.com
MAIL From: TheBoss@example.com
RCPT To: me@my.email.address 
DATA
Subject: Think we're insecure...
I have a feeling our mail server is being abused...
.
QUIT

Now if you check your email you will notice an email from "TheBoss"
and you'll see how easy it is to spoof an email.
For more examples using telnet, see the dovecot documentation
TestInstallation.txt

Module Code and Module Title Title of Slides

 Problem Solving
tail /var/log/messages
– Make sure variables are sound and correct on start up
– Make sure dns resolving and lookups are correct when
mail is received

tail /var/log/maillog
• SMTP and POP3 communications
• IP address and time stamp
– Make sure no errors have been reported
• Make sure you created a mail inbox in /var/mail
• Make sure mail is being placed in user /var/mail/username
• check the mail queue

Module Code and Module Title Title of Slides

 Problem Solving: syslog
/etc/syslog.conf
> Specifies how and which events logs
• Monitor SMTPD connections:
egrep “[^s]connect from|client=“ /var/adm/syslog
• Monitor bounced messages:
grep status=bounced /var/adm/syslog
• Monitor deferred messages:
grep status=deferred /var/adm/syslog

NOTE: these logfile locations are just examples

Check your conf files for exact locations

Module Code and Module Title Title of Slides

 User Accounts
• Unique username, UID number, and password
– Stored in system password database
• /etc/passwd /etc/shadow
• /etc/group /etc/gshadow
• Create login directory for user (home)
• Specify initial shell program
• Set up standard initialisation files and login environment
– defaults are usually copied from /etc/skel
• Usually done using adduser / useradd command

useradd -c “Bill Gates” -u 1001 -g msoft

-d /home/billg -m -k /etc/skel -s /bin/bash billg

Module Code and Module Title Title of Slides

 Ownership of files and processes

• UID Zero: root (superuser)

• UIDs mapped in /etc/passwd
• GIDs mapped in /etc/group
File permissions Read, Write, Execute
4+2+1=7
Owner, Group, World
644 means what?
755 means what?

Module Code and Module Title Title of Slides

 Email only accounts
• Creating a standard user account with a false shell stops that
account from being able to log into the system (a Linux login),
but still allows them to use the system for the purpose of
sending and receiving email.
• Creating a user account using the following example, is the
easiest method for creating email only accounts for friends
and family who do not require login access to the server.

useradd -c "Alice Jones" -s /sbin/nologin alice

What does this mean in terms of ownership of files and processes?

What does this mean in terms of maintenance/administration?

(revisit the configuration decisions discussed earlier)

Module Code and Module Title Title of Slides

 System Users

• postfix / dovecot / nobody

• vmail:vmail

Module Code and Module Title Title of Slides

 LDAP-Based Mail Routing -
Concept
• LDAP or YP (NIS) server
– Central directory services
• better than trying to share the same passwd file
– User names and passwords
– email addresses
– email aliases
• Mail is delivered to user’s preferred location via SMTP
• Preferred location is determined by LDAP lookup
– Similar to, but not the same as MX lookup
• Mail to non-existent users is rejected at mail router;
aliases are processed as special cases

we’ll talk more about LDAP and try it out soon

Your project demonstration MUST show a working LDAP server
Module Code and Module Title Title of Slides
 This week in the lab

• We do some manual configuration for the mail system

– Mostly creating system and user accounts

• This plus the Marked Assignment will finish off the

“foundation system” you need to demonstrate next week
– See Moodle for details

• Start thinking about the group project

– Who you will work with
– What enhancements you will implement

Module Code and Module Title Title of Slides

Module Code and Module Title Title of Slides