The problem of backscatter, part 16 - BATV and SPF

We've seen that BATV is one of the better mechanisms to stop backscatter, the question now is how do we use it?  What stuff does it potentially break?

Some of the commenters in my other posts have alluded to it when they have said that you can't use BATV unless you have an SPF policy that dictates a hard fail on your outbound mail.  Why is that?

The reason is that if you don't know where your mail is coming from on outbound, you can't necessarily say it didn't come from you if it isn't signed in a bounce.  For example, if your SPF record is this:

v=spf1 ip4: -all

...then you know that all your outgoing mail comes only from those IP addresses and so do all your recipients, and recipients should hard fail (reject) all mail that claims to come from you that is outside of those IPs.  Since you know which IPs you send mail from and no others, you can also know that you always sign mail from those IPs as well.  Thus, a bounce message that isn't signed means that it didn't come from those IPs; you can "hard fail" the bounce message.  It's kind of like a secondary SPF check.

However, suppose your SPF record is one of the following:

v=spf1 ip4: ~all

v=spf1 ip4: ?all

In the former case, a soft fail ~all means that if mail comes from you but is outside your IP range, then it probably, but not necessarily, didn't come from you.  Accept it but mark it.  In the latter case, a neutral ?all, it means that if you (the receiver) get a mail from those IPs then it definitely came from us, but if not then it may or may not.  For you see, we're not entirely sure which IPs we use to send outbound mail.  Thus, we neither confirm nor deny anything about mail claiming to come from us that is outside those IP ranges.

And therein lies the problem for these two cases.  If you can't say for sure where mail from your domain comes from, you can't be sure that all of your outbound mail is signed.  If you can't say that all of your outbound mail is signed, then you can't reject it using BATV.  An unsigned message doesn't mean that the message didn't come from you -- it says so right there in your SPF policy.   You'd have to parse email content in order to figure out where it came from.  If you could do that, then you could have some conditional logic implemented because you know that some set of IPs you do sign on the outbound.  That is starting to get a little convoluted, however, and it's prone to failure because you have to rely on recipient MTAs to send back all of the necessary received headers.  But then again, if you could do that (figure out where it came from by parsing and trusting the original received headers), why do you need BATV?

Alternatively, what you could say is if a bounce is signed and passes a BATV check, accept the message without further filtering.  Conversely, if it isn't signed and you know that it is a bounce, filter it more aggressively (ie, don't de-spamify a message classification).  The problem is that this takes us back to the problem of false positives; on the other hand, you'll probably have fewer FPs anyhow because the ones you want are probablysent from your known good IP range.

I'm not really a big fan of filtering messages more aggressively, I'm just saying that you could do it this way if you had soft fail or neutral SPF policies.