Sender authentication part 6: The basics of SPF

In our previous posts on sender authentication, we were introduced to the concepts of SMTP, internet headers and how spammers will try to spoof headers.

One of the weaknesses of SMTP is that the sender can assign any email address as the Envelope sender and specify another email address as the sender in the message headers. Thus, if a message gets past your spam filter and hits your inbox, you might be tempted to think that the message is coming from someone who, in fact, did not really send the message. Most spam today carries fake email addresses so you can't really complain to the one who is purportedly sending you the message.

Fortunately, spam fighters have a tool to combat this.  The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent sender address forgery. More precisely, the current version of SPF — called SPFv1 or SPF Classic — protects the envelope sender address, which is used for the delivery of messages.

We will get into more details in a later post, but here's how it works in a nutshell:

1. A domain registers the IPs that they permit to send email. Let's say that our example domain is They decide to say "Okay, the only IPs that are allowed to send email from this domain are to inclusive."

2. An email server,, receives an email and the envelope sender says the message came from The mail came from

3. looks up's SPF record and sees that they permit through The transmitting IP is and is within this range, therefore this email did indeed come from

In this case, this would be an SPF pass because the IP that sent the mail is permitted to send mail from However, suppose that the next message arriving at is a little bit different.

1. The message was sent by, and the transmitting IP is

2. looks up's SPF records and sees that the transmitting IP is not within the range declared by that is permitted to send email.

3. In this case, the message fails an SPF check (this is known as an SPF Hard Fail); the interpretation is that is being spoofed and the message is routed to the user's spam quarantine.

What the email server does with the message from that point forward is up to them, but the point is we have a case where an email claims to be from but is transmitting from an IP that is not permitted to send email from that domain.

Those are the basics of SPF, but by no means is that all there is to it. Besides looking up the SPF records for the sending domain, comparing it to the transmitting IP and returning SPF Pass or SPF Hard Fail, there are other possibilities:

1. SPF Hard Fail - the transmitting IP is not permitted to send mail for that domain.

2. SPF Soft Fail - the transmitting IP does not meet a domain's strict definition of legitimacy, but the domain cannot classify the message as a forgery for certain.

3. SPF Pass - the transmitting IP is permitted to send mail for that domain.

4. SPF None - the domain does not publish SPF records.

5. SPF Neutral - the SPF check doesn't know and is returning no opinion.

6. SPF Error - the mail server received an error when looking up a domain's SPF record.

7. SPF Unknown - the mail server could not look up the domain's SPF record, possibly the domain's SPF record is improperly configured.

We will examine some of the syntax of SPF records in a future post (which will explain the difference between Hard and Soft fails). For now, look at number 4, SPF None.

Suppose that gets a message from from (envelope sender). The From address in the message header says and the subject line says "Please confirm your identity". This, of course, is a classic phishing spam. With SPF in our arsenal, we can grab sending IP and compare it to the SPF records from Yahoo. If the IP fails the SPF check, we can filter the message as spam. If it passes the check, we can contact Yahoo and let them know this user is using their services to send spam.

But herein lies the rub: does not publish SPF records. Remember that we perform an SPF check on the envelope sender, which is The receiving email server looks up the SPF records for, sees nothing there, and returns an SPF None. The message continues to sail through the reputation check and if the content filters don't grab it, this message arrives at the user's inbox with as the sender in their email client. Conceivably, this can fool some users (and it does all the time, judging by the number of phishing false positives I see).

And that's the conumdrum, SPF only works with domains that publish SPF records, and most domains don't publish them. I don't know the numbers but last I heard, less than 25% of all domains do. The whole point of domains publishing their SPF records is to prevent themselves from being spoofed in the envelope sender, so if domains don't get on board with SPF, this check won't work.

Intelligent spammers who phish (and even generic spammers) will spoof sending domains without SPF records. Because those checks will return an SPF none, a spam filter cannot drop the message right there (assuming that the client wants auto-reject SPF Hard Fails). By getting past that, they only need to get past the content filters. That, in itself, is a challenge but it's one less hoop they have to jump through.