Block spam and phishing using SPF records
The Domain Name System (DNS) is a critical and foundational service for all forms of modern networked computing including the Internet. DNS is implemented as a hierarchical, distributed database on servers called DNS servers or name servers. This database contains mappings of DNS domain names to various types of data such as the Internet Protocol (IP) addresses of hosts and other nodes on the network.
Because proper functioning of DNS is so critical to the operation of your network, failure or misconfiguration of DNS can lead to serious problems. DNS is therefore a primary target for many types of network attacks, and spammers and phishers in particular often try to make use of it to target an organization's messaging services. For shops that use Microsoft Exchange or Office 365 as their messaging platforms it's therefore important to properly configure your DNS servers to block spam and phishing attacks against your organization.
One important way of mitigating both spam and phishing in Exchange/Office 365 environments is to make use of Sender Policy Framework (SPF) records which are a special type of DNS record that identifies which mail servers are allowed to send email on behalf of your company's domain. To help us understand this subject better, I've asked Andrew Perchaluk, a Senior Systems Administrator at the University of Manitoba in Winnipeg, Canada to provide us with some insight and tips from his own experience managing Active Directory environments. Andrew is a husband, father, and dog lover who has been working in the Information Technology industry for almost 20 years and who enjoys sharing his experiences with others in the IT pro community. For more information about Andrew see his LinkedIn profile. You can also follow him on Twitter. Let's now hear what Andrew has to say on this topic.
Understanding SPF records
Whether you're responsible for your company's email at the CIO, Director, Manager or System Admin level you've likely been involved in or heard about changes that have been made with your message hygiene systems to crack down on SPAM and Phishing emails over the years. A couple other things you can do to further crack down are:
- Set up an SPF record in your company's DNS and make sure it's valid
- Set up incoming SPF checking by your message hygiene product
An SPF record is a type of Domain Name Service (DNS) record that identifies which mail servers are permitted to send email on behalf of your domain. The purpose of an SPF record is to prevent spammers who are trying to send messages with forged from addresses at your domain (known as spoofing). Clients prefer that the emails from the vendors show up as coming from a @umanitoba.ca email addresses instead of the vendors email address.
Is your SPF record valid?
Many companies SPF record is not, when writing this article I found a number of companies out there that had errors present in their SPF records. The errors are typically that the record is too large or has too many lookups.
According to RFC 7208 an SPF record can only have 10 lookups. Also you'll find some materials out there that say an SPF record can only be 255 characters, this is really the limitation of a TXT record however there is a workaround. RFC 4408 3.1.3 says a single text DNS record (either TXT or SPF RR types) can be composed of more than one string.
If you're unsure how to tell if the SPF record you have is set up correctly validate it first! SPF syntax can be a little tricky to get the hang of so take a look at this reference for a detailed explanation of available syntax http://www.openspf.org/SPF_Record_Syntax
Once you've got your SPF record in place there are many tools out there that you can use to validate a couple I like are:
- Dmarcian SPF viewer - https://dmarcian.com/spf-survey
- SPF Record Testing Tools - http://www.kitterman.com/spf/validate.html
- Wise Tools - http://tools.wordtothewise.com/authentication
I purposely set up an invalid SPF record (v=spf1 include:surveymonkey.com a:cal1psmtp.desire2learn.com ?all ) at perchaluk.ca and ran the Dmarcian SPF domain survey against it. I received the following feedback:
- PTR mechanisms SHOULD NOT be used and cannot be resolved using this diagnostic tool. More info at https://space.dmarcian.com/ptr-mechanisms-in-spf-records/.
Receivers are unable to use this SPF record to determine authenticity:
- Too many DNS-querying mechanisms (count=30).
- 30 DNS-querying mechanisms/modifiers required to resolve record (maximum of 10 allowed).
- 143 authorized netblocks (727,700 authorized individual IPv4 addresses). Authorized netblocks produce SPF "pass" results (as opposed to "neutral", "fail", or "softfail").
Because SPF lookups will also perform lookups on the external companies you have included in your record and that will count against your 10 lookups you have to be cautious. That's the reason for the errors above even though it seems like my SPF record only has to perform a couple lookups but when the lookup checks and expands the surveymonkey.com SPF record
The nice thing about this tool is it shows you the breakdown of your SPF record and includes the external DNS queries. The first screenshot below shows the output from the SPF viewer tool at dmarcian.com, it allows you to expand all the combined lookups in your SPF record (including external companies). This is very useful in troubleshooting the "Too many DNS lookups" error, you can then see which companies that you've included in your SPF might be causing issues for you.
The other nice feature is that the tool will give some guidance on how you can convert your SPF record into one with a valid format, so in the case of my invalid record for perchaluk.ca, it took it and changed it from using PTR and MX to IP based records. It also split the records into 5 separate strings which keeps things under the 10 DNS lookups and 255 character limit. This next screenshot shows the results of the record flattening tool at dmarcian.com, this will take you invalid SPF record and convert it to multiple IP4 based TXT records. It removes duplicate netblocks, collapses overlapping netblocks and presents you with a valid SPF record. In my case I had to create the 6 TXT records listed to change the SPF record for perchaluk.ca to a valid RFC compliant record.
Then after updating the DNS records for perchaluk.ca and creating the 6 SPF records listed above I am left with a valid and useable SPF record. Running a follow-up query now gets a passing grade as follows:
No warnings or errors
- 5 DNS-querying mechanisms/modifiers required to resolve record (maximum of 10 allowed).
- 130 authorized netblocks (727,693 authorized individual IPv4 addresses). Authorized netblocks produce SPF "pass" results (as opposed to "neutral", "fail", or "softfail").
Set up incoming SPF checking
Now that you have a valid SPF record you can configure your mail system to perform SPF checking. For Office 365 you would simply set your SPF record to on under advanced options. The next screenshot shows how to turn on SPF checking for Office 365 the image is from the Office 365 Admin portal, go to the Exchange Admin center | Protection | Spam Filter, then edit the default or other policy. Then go to Advanced options, the only option is for hard fail so you would really want to make sure your SPF record is correct and captures all required senders for your O365 tenant.
Start with logging or tag only
If you use a mail security product like Cisco IronPorts or Barracuda email security you have even better options. I currently use Cisco IronPorts for our message hygiene so will explain some options here.
If you are a company with a number of older email domains leftover you may have seen how these old domains can be a large source of SPAM and Phishing. I started by enabling SPF checking for incoming mail but to start with only for a couple older email domains for example @oldmail1.perchaluk.ca and @oldmail2.perchaluk.ca addresses to start with, I found there was a lot of incoming spam and spoofing from these addresses.
To do this we would do the following on the IronPorts
- Enable SPF checking on the HAT mail flow policies called THROTTLED and ACCEPTED (it can be enabled on others if required).
- This will evaluate SPF when accepting mail.
- Then create 4 incoming content filters (2 per mail domain, two for @oldmail1.perchaluk.ca, two for @oldmail2.perchaluk.ca) which will do the SPF check and apply to the appropriate incoming mail policy.
The content filters would look like
- Condition: SPF fail
- AND condition: "Other Header", "From:" "ends with @oldmail1.perchaluk.ca
- Make sure to select Apply rule "only if all conditions match"
- Action: log
- Note: we need a CF for @oldmail1.perchaluk.ca and a separate one for @oldmail2.perchaluk.ca because you can't have AND + OR statements in a CF
- Next create another content filter for @oldmail1.perchaluk.ca following the steps above
- Except use condition "Envelope sender" instead of other header
The next screenshot is from the Cisco Enterprise security appliance (IronPorts), you can create the required content filters under Mail Policies | Incoming content filters | Add filter. This one will check the From field in the headers for the specified mail domain. It's always recommended to start by only logging SPF checks if possible.
This next screenshot is also from the Cisco Enterprise security appliance (IronPorts), you can create the required content filters under Mail Policies | Incoming content filters | Add filter. This one will check the envelope sender for the specified mail domain. It's always recommended to start by only logging SPF checks if possible.
Switch to quarantine or blocking
After you've spent some time monitoring the logs and are comfortable with the Content Filters, you would change the action from log to quarantine. On a Barracuda the change would be from tagging to blocking. This final screenshot is from the Cisco Enterprise security appliance (IronPorts), you can create the required content filters under Mail Policies | Incoming content filters | Add filter. After you've reviewed the logs and corrected any issues that may have showed up change the actions on the previously created send filters from logging to quarantine.
If all goes well with these first 2 mail domains you can then look at adding additional CF's for other mail domain's you own in the in the future (ie: @perchaluk.ca, etc).
Still got questions about Office 365?
If you have any questions about administering Microsoft Office 365, the best place to ask them is the Office 2016 and Office 365 Pro Plus forum on TechNet. If you don't get help that you need there, you can try sending your question to [email protected] so we can publish it in the Ask Our Readers section of our newsletter and see whether any of the almost 100,000 IT pro subscribers of our newsletter have any suggestions concerning your problem.