Ravenstar's Email FAQ's

SPF, DKIM, DMARC Explained

The purpose of this post is to explain 3 technologies that are increasingly being used in email.  It aims to look at the pro's and cons of the technologies as well.

I'll cover SPF and DKIM and then move on to DMARC as the latter actually requires the former two technologies in order to work.

General Introduction

SPF, DKIM and DMARC are all designed to protect domains emails from being spoofed by third parties.  They are are all implemented at some level by adding DNS records by the domain administrators.  Although DKIM does also use cryptographic methods to sign an email to verify the emails integrity.

Broadly speaking.

SPF announces the mail servers that should be delivering mail for a domain DIRECTLY to the destination mail servers.

DKIM embeds a signature in the mail to prove that the mail is genuine.

DMARC provides instruction to the receiving server, as to what to do should a mail fail SPF and/or DKIM.

Message Envelopes

Before going any further I also need to discuss message envelopes as these are important in SPF.
Just like real mail.  When an email is sent  a message envelope is constructed.  This is similar to when you address an envelope in the real world, except that your email program does this automatically.

The message envelope is constructed by sending two commands to the mail server.

mail from: <senders-address>
rcpt to: <recipients-address>

While in most basic e-mails the senders-address will be the same as the From: address in the email and the recipients-address will be the same as the To: address in the email - there are times when this is not the case.  This is something that spammers know and can take advantage of.

The important thing to note is that mail is delivered to the <recipients-address> and not to the To: address in an email.  Likewise bounced mail is returned to the <senders-address> and not the From: address.

SPF was designed to use the domain in the <senders-address> when evaluating if mail is coming from an appropriate source.

DKIM and DMARC on the other hand both use the domain in the From: address.

SPF - SENDER POLICY FRAMWORK

As discussed above SPF is a way for domain administrators to let the world know which servers deliver their outbound mail.  The administrator publishes a text record under the bare domain.

For example the domain example.com might publish the following TXT record

"v=spf1 a -all"

In this case the record states that the mail server is found on the same address as indicated by the A record lookup for example.com.  The -all at the end indicates that no other addresses should be used.

So if someone sends me a mail.  My inbound server looks at the <senders-address> discussed above.  It takes the domain and then does a DNS lookup to see if an SPF record exists.

If so it then does another DNS lookup for the A record to get the IP address.

Finally it does a comparison of the IP address and the IP address of the server that it's talking to.  If the two match then the SPF passes.  If the two differ then  the server continues to evaluate the next entry on the SPF string.  In this case the next entry is -all which tells it that all other addresses should be failed.

The SPF record Syntax is discussed in more detail here

http://www.openspf.org/SPF_Record_Syntax

Possible Final Outcomes

The last parameter provides for a number of possible outcomes should the mail server not match any addresses provided by the SPF record.

+all - pass the mail.  Effectively saying "I don't see any value in SPF."
?all - neutral result (this is the default)  It allows the mail through without saying anything about it's authenticity.
~all - soft fail.  Mark the mail as failed but don't act on the failure.  Useful when testing SPF records.
-all - Hard fail mark the mail as failed and act appropriately.

SPF and Email Forwarders.

Because SPF tells a server which mail servers should be directly delivering a domains mail to it.  SPF fails when mail is passed to it's final destination via a forwarding service.  One possible suggestion to overcome this was that Forwarders rewrite the <senders-address> when forwarding the mail.  While some forwarders do this.  Many do not meaning SPF on it's own only has real value when mail is being delivered directly to the recipient's email address.

Server response to SPF fails.

Up til the creation of DMARC, there was no mechanism in place for telling recipient servers what to do if an SPF check failed.  It was left to the administrators of the receiving domain's email servers to decide the best policy.  Most providers record the result of the SPF check in the headers, although Virgin does not at this time.  Other possible solutions were to quarantine failed mail, deliver it to the spam folder or reject it altogether.

SPF means attitudes of Senders needs to change.

One common thing a lot of people do is use the outgoing servers of their current ISP to send mail for a number of different email addresses.  In the past this was not an issue.  In fact it's the way the mail standard was built.  Unfortunately the reality of the need to protect domains from spammers has changed that.

When sending mail from any email address - senders need to use the smtp servers associated with that email address - ESPECIALLY when the address has an SPF record deployed.  Failure to do so will mean the mail fails SPF checks and is more likely to be marked as spam in the first instance.  If the domain has a DMARC reject policy deployed, the mail is likely to be rejected on those grounds as well.

DKIM - DOMAINKEYS IDENTIFIED MAIL

DKIM was an extension of a similar scheme created by Yahoo called DomainKeys.  DKIM proves the authenticity of an email by signing it with a DKIM Header.  This Header contains the following.

• A key to identify which record to use.
• Information as to any headers that are to be used to authenticate the mail.  As well as whether to use relaxed or canonical checking.
• A hash of the text of the bodies and any selected headers.

DKIM works in the following way.  The server administrator produces a pair of Cryptographic keys.  The Private key is used by the email server and the public key is published in a TXT record via DNS

When a mail is sent through the outbound server the text of the mail is parsed and a DKIM header added to the mail using the private key and the information in the body and chosen headers, as well as an index which tells it which particular DKIM record it should be using.

When the mail is received by an exchanger it looks at the DKIM record and then uses the domain in the From: address to look up the DKIM entry index._domainkey.<domain_name>  It will then parse the body and selected headers to determine if the signature in the DKIM header is valid.

Provided no changes have been made to the relevant parts of the mail, these checks will pass.  It is even possible to pass the mail through a forwarding service and the mail will still pass PROVIDED the forwarding service does not add anything to the body of the mail.

I say this because I've seen at least one example on these Forums where a mail has been rejected because it was being sent to an institution that forwarded the mail, but before doing do added their standard disclaimer to the bottom of the mail.  This caused the mail to fail DKIM, and as the sender had a DMARC reject policy set up, the mail was bounced by Virgin.

Like SPF, DKIM alone does not tell the recipient domain what to do when a mail fails checks.  It simply offers a pass/fail/neutral result.

DMARC - Domain-based Message Authentication, Reporting & Conformance

DMARC builds on the above two protocols.  It allows the reporting of SPF and DKIM failures as well as providing a means to announce a policy that advises recipient servers what they should do with failed mail.

Again this relies on DNS entries to work effectively.  And again uses the address in the From: address of an email in order to look up the DMARC record for a domain.

Lets's have a look at a real world DMARC entry - in this case yahoo.com

The DMARC record is published as _dmarc.<domain>

> set type=txt
> _dmarc.yahoo.com
Server:  cache1.service.virginmedia.net
Address:  194.168.4.100

Non-authoritative answer:
_dmarc.yahoo.com        text =

        "v=DMARC1; p=reject; pct=100; rua=mailto:dmarc_y_rua@yahoo.com;"

In this the record shows.

1. Version - v=DMARC1
2. Policy - p = reject in this case mails that fail DKIM or SPF should be bounced.
3. Percentage of mail to check - pct=100 in this case all of it.
4. Where to send aggregate failure reports.

Note that DMARC can also be used to send reports on individual mail failures as well.

DMARC is useful as it standardises the outcome of SPF and DKIM and puts the choice as to what to do with the failed mail back in the hands of the sending domain.  It should be noted that

DMARC on it's own is pointless.
DMARC with SPF alone will encounter problems where mail ends up being sent via forwarders.

For that reason IMHO DMARC is best deployed when the sending domain utilises BOTH DKIM and SPF.

Ravenstar68