Uncovering the Exchange 2007 Edge Transport Server (Part 5)

If you would like to read the other parts in this article series please go to:

The Content Filtering Agent

The Content Filter agent can be considered the next generation of the Intelligent Message Filter (or IMF version 3), which most of us know from Exchange Server 2003 (version 2 came with Exchange 2003 SP2). This means that the Content Filter is based on the SmartScreen technology which originally was developed by Microsoft Research. When an E-mail message is received by an Edge Transport server with the Content Filter agent enabled, it will evaluate the textual content of the messages and then assign the message a spam confidence level (SCL) rating based on the probability the message is spam. This rating is stored as a message property called a Spam Confidence Level (SCL) rating. The Content Filter is regularly updated using the Anti-spam Update Service (Windows Update) in order to ensure that it always contains the most up to date information when running. Since the Content Filter is based on the characteristics of many millions of messages (Hotmail among others are used to collect the necessary information about both legitimate as well as spam messages), it recognizes both legitimate messages and spam messages. The Content Filter can very precisely determine whether an inbound e-mail message is a legitimate message or spam.

The Content Filter can also, via spam signatures, analyze messages for phishing characteristics. And if the message is a phishing attempt, the Content Filter agent will stamp it with a property before delivering it to the recipients Inbox. When it is delivered Outlook 2007 will render it differently and warn the user that this most likely is a phishing attempt. When the message is viewed in Outlook 2007 all content will be flattened, any links will be disabled and no images will be loaded.

Just as was the case with IMF in Exchange Server 2003, you can, with the help of the Content Filter, assign a spam confidence level (SCL) rating to the message flowing in to your organization. The Content Filter stamps the messages that it inspects with an SCL property (actually a MAPI property), with a value between 0 and 9. As you can see in Figure 1 below you can, depending on how a message is rated. either delete, reject or quarantine it to a specified mailbox.


Figure 1: Action Tab on the Content Filtering Property Page

If a message equals the SCL delete threshold, the message will be deleted without notifying the sending server. If the message equals the SCL reject threshold the message will also be deleted, but a rejection response will be returned to the sending server. If a message equals the SCL quarantine threshold, the message will be sent to the E-mail address specified in the Quarantine mailbox e-mail address: field. Bear in mind that before a message can be quarantined you need to create and configure a mailbox that should be used for this purpose. In order to do so perform the following steps:

  1. First create a new mailbox called Quarantined Messages or similar.

  2. Depending on how many recipients as well as how many messages are received by your Exchange organization, configure a reasonable quota for this mailbox.

  3. Set up delegation if you are going to open the mailbox as an additional mailbox under your primary mailbox account.

  4. On the Edge Transport server open the Exchange Management Shell, then type: Set-ContentFilterConfig -QuarantineMailbox <SmtpAddress>, then hit Enter.

All quarantined messages will now be sent to the specified E-mail address, so be sure to check it for any false positives on a regular basis. When you find a false positive you can resend it to the original recipient by opening the message, then clicking Resend.

In addition to this you can create a list of words and/or phrases that will not be blocked no matter the SCL rating of the particular message (the Content Filter will assign a SCL rating of 0 to messages including these words and/or phrases). You can also create a list of words and/or phrases that should be blocked no matter the SCL rating (see Figure 2).


Figure 2: Custom Word List on the Content Filtering Property Page

If you for some reason do not want to block any messages destined for a particular SMTP address, you can add the address to an exceptions list (see Figure 3).


Figure 3: Exceptions List on the Content Filtering Property Page

Safelist Aggregation

The Content Filter agent also includes another anti-spam feature, which is not visible in the Exchange Management Console GUI. It is called safelist aggregation and is a feature which collects data from the Safe Senders and Safe Recipients lists, which can be found under the Junk E-mail Options in Outlook 2007 (see Figure 4).


Figure 4: Safe Senders List in Outlook 2007

The E-mail addresses and/or domains which the end-users in your Exchange organization have added to the Safe Senders and/or Safe Recipients list are stored on the respective mailbox server on which a mailbox is located, and can from here be pushed to Active Directory directory service, where the lists can be stored on each user object. If you make use of the EdgeSync service these lists will, as part of the recipient data, be replicated from Active Directory to the ADAM store on the Edge Transport server.

In order to reduce the amount of false positives on the Edge Transport server, the Content Filter agent can let the E-mail addresses and domains configured on the Safe Sender list be allowed to pass through to the end-users mailbox without additional processing by the rest of the filtering agents by using safelist aggregation. If you enable Also trust e-mail from my Contacts under the Safe Senders tab in Outlook 2007 (see Figure 4), all Outlook Contacts in the users mailbox will be allowed to pass through the filtering agents as well, pretty neat right?

Even though you have enabled the Content Filtering agent, you still need to enable and configure the safelist aggregation feature before you can make use of it. In order to do so perform the following steps:

  1. Logon to the Exchange 2007 server which has the Mailbox server role installed then open the Exchange Management Shell (EMS).

  2. In order to read the Safelist collection from each user’s mailbox and then hash and write it in the respective user objects in Active Directory, you will need to run the Update-Safelist CMDlet. When using the Update-Safelist CMDlet provide the identity for the mailbox you want to run the CMDlet on. Since we want to run the Update-Safelist CMDlet on all Mailbox users on the Mailbox server, we will need to make use of piping. To run the Update-Safelist for all mailbox users type: "get-mailbox | where {$_.RecipientType -eq [Microsoft.Exchange.Data.Directory.Recipient.RecipientType]::MailboxUser } | update-safelist" then hit Enter.

  3. Since the Update-Safelist CMDlet is a onetime only command, you would need to use the Windows Scheduler in order to schedule the CMDlet to run, let’s say one every 24 hours. In order to do so create a batch file with the following code:
    "C:\Program Files\Windows Powershell\v1.0\Powershell.exe" -psconsolefile "C:\Program Files\Microsoft\Exchange Server\bin\exshell.psc1" -command
    "Get-Mailbox | where {$_.RecipientType -eq [Microsoft.Exchange.Data.Directory.Recipient.RecipientType]::MailboxUser } | Update-Safelist".

  4. Now save the batch file as Update-Safelist.bat or something similar (remember to change Notepad to all files instead of txt files).

  5. Now schedule this batch file to be run every 24th hour (for example at 00.00). To do this open a command prompt window (or use the Windows Scheduler which can be found in the Control Panel) and type: AT 00.00 /every:M,T,W,Th,F,S,SU cmd /c “C:\Update-Safelist.bat” then hit Enter

  6. In order to see whether the Update-Safelist has updated the respective Active Directory user objects, you can check the msExchSafeRecipientsHash and msExchSafeSendersHash attribute for a couple of user objects using ADSI Edit or a similar tool. If these attributes has a value of <Not Set>, they haven’t been updated, but if they instead has a value similar to 0xac 0xbd 0x03 0xca, the user objects have been updated.

Note:
In order to use ADSI Edit you need to install the Windows Server 2003 Support Tools on the respective Exchange 2007 server.

To see whether Safelist aggregation works as expected on the Edge Transport server, try to add a custom word or phrase to the Custom Words block list which is found on the Property page of the Content Filter. Now add the E-mail address of the private E-mail account (such as a Hotmail, etc.) to the safe senders list of your mailbox in Outlook 2007. Finally send an E-mail message, containing the word or phrase you added to the block list, to your Exchange 2007 Mailbox user account.

If the message appears in your mailbox the Safelist aggregation feature works as expected.

Outlook E-mail Postmark Validation

In addition to the Safelist aggregation feature, the Content Filter agent includes one more feature that will help reduce the amount of false positives in your Exchange organization. The feature is called Outlook E-mail Postmark Validation which is a computational proof that Outlook applies to all outbound messages in order help recipient messaging systems to distinguish legitimate E-mail messages from Junk E-mail. With Outlook E-mail Postmark Validation enabled the Content Filter agent will parse all inbound messages for a computational postmark header. If a valid as well as solved computational postmark header is present in a message, it means that the client computer that generated the message solved the computational postmark. The result of a postmark validation will be used when the overall SCL rating for an inbound message is calculated.

Note:
If no computational postmark header exists or if the header is invalid, the SCL rating will not be changed. On a default installation of the Edge Transport server role the Outlook E-mail Postmark Validation feature is enabled by default, but to verify the feature indeed is enabled on your system, you can open an Exchange Management Console and type Get-ContentFilterConfig (see Figure 5).


Figure 5: Content Filter Configuration Settings

If the feature for some reason is set to False, you can enable it by typing: Set-ContenFilterConfig –OutlookEmailPostmarkValidationEnabled $True, then hit Enter.

The Attachment Filtering Agent

As most of you are aware, Exchange Server 2003 did not include a functionality that let you filter out specific attachments, instead you either had to create your own SMTP OnArrival Event Sink, use a 3rd party product or strictly rely on the OWA 2003 and Outlook 2003 attachment control feature, but since you really should filter out unwanted attachment types on an SMTP gateway in your perimeter network (aka DMZ or screened subnet) before they arrive to your internal network, the last two options are not recommend. I would say that an attachment filtering mechanism should have been a native feature in Exchange a long time ago, but finally the wait is over as the Edge Transport server in Exchange Server 2007 lets us do attachment filtering at the server level (hooray!). We now have the possibility of filtering out messages based on attachment file name, file name extension, or file MIME content type. We even have the choice of filtering out both the message and the attachment or just strip the attachment. We can even choose to delete both the message and attachment silently, meaning that both will be deleted without notifying the sender of the message.

Side note:
In recent years there has been increasing focus on deploying messaging environments, where each individual E-mail message is either digitally signed or encrypted or even protected using Information Rights Management (IRM). Here in Denmark, where I live, governmental institutions have a strong desire for protecting the messages while they are in transit. If you are doing the same in your organization or are planning on doing so, you should bear in mind that stripping an attachment from either a digitally signed, encrypted or IRM protected E-mail message will invalidate the message, so that it becomes unreadable. One way to solve this problem, focusing on digitally signed or encrypted messages, is to put up some kind of blackbox, which takes care of signing and encrypting the messages after the attachment filter has processed them. The company I work for has got such a product which is becoming more popular here in Denmark.

The Attachment Filtering agent applies right after the Content Filtering agent, and can be configured using the Add-AttachmentFilterEntry CMDlet. Unfortunately there is no way to configure Attachment Filtering via the Exchange Management Console GUI, you will have to do so using the Exchange Management Shell. I do not really know why this feature has not been included in the GUI, but my guess is that the Exchange Product team did not have the time to integrate the feature in the GUI. If this is the case we can expect it to be included in Exchange Server 2007 Service Pack 1, but only time will tell.

Before we start to configure the Attachment Filter agent, we first need to make sure the agent is enabled. In order to do so you will need to open the Exchange Management Shell (EMS) and type Get-TransportAgent. On a default installation of an Edge Transport server this agent should be enabled by default, but if it for some reason is disabled you can enable it by typing Enable-TransportAgent -Identity "Attachment Filtering Agent" then hitting Enter.

Now that the agent is enabled let’s type Get-AttachmentFilterEntry | FL, then press Enter. This will give us a list of all file name extensions and content types that the attachment filtering agent can filter on (Table 1).

Type

Name

Identity

ContentType

Application/x-msdownload

ContentType:application/x-msdownload

ContentType

Message/partial

ContentType:message/partial

ContentType

Text/scriptlet

ContentType:text/scriptlet

ContentType

Application/prg

ContentType:application/prg

ContentType

Application/msaccess

ContentType:application/msaccess

ContentType

Text/javascript

ContentType:text/javascript

ContentType

Application/x-javascript

ContentType:application/x-javascript

ContentType

Application/javascript

ContentType:application/javascript

ContentType

x-internet-signup

ContentType:x-internet-signup

ContentType

Application/hta

ContentType:application/hta

FileName

*.wsh

FileName:*.wsh

FileName

*.wsf

FileName:*.wsf

FileName

*.wsc

FileName:*.wsc

FileName

*.vbs

FileName:*.vbs

FileName

*.vbe

FileName:*.vbe

FileName

*.vb

FileName:*.vb

FileName

*.url

FileName:*.url

FileName

*.shs

FileName:*.shs

FileName

*.shs

FileName:*.shb

FileName

*.sct

FileName:*.sct

FileName

*.scr

FileName:*.scr

FileName

*.scf

FileName:*.scf

FileName

*.reg

FileName:*.reg

FileName

*.prg

FileName:*.prg

FileName

*.prf

FileName:*.prf

FileName

*.pcd

FileName:*.pcd

FileName

*.ops

FileName:*.ops

FileName

*.mst

FileName:*.mst

FileName

*.msp

FileName:*.msp

FileName

*.msi

FileName:*.msi

FileName

*.ps11xml

FileName:*.ps11xml

FileName

*.ps11

FileName:*.ps11

FileName

*.ps1xml

FileName:*.ps1xml

FileName

*.ps1

FileName:*.ps1

FileName

*.msc

FileName:*.msc

FileName

*.mdz

FileName:*.mdz

FileName

*.mdw

FileName:*.mdw

FileName

*.mdt

FileName:*.mdt

FileName

*.mde

FileName:*.mde

FileName

*.mdb

FileName:*.mdb

FileName

*.mda

FileName:*.mda

FileName

*.lnk

FileName:*.lnk

FileName

*.ksh

FileName:*.ksh

FileName

*.jse

FileName:*.jse

FileName

*.js

FileName:*.js

FileName

*.isp

FileName:*.isp

FileName

*.ins

FileName:*.ins

FileName

*.inf

FileName:*.inf

FileName

*.hta

FileName:*.hta

FileName

*.hlp

FileName:*.hlp

FileName

*.fxp

FileName:*.fxp

FileName

*.exe

FileName:*.exe

FileName

*.csh

FileName:*.csh

FileName

*crt

FileName:*.crt

FileName

*.cpl

FileName:*.cpl

FileName

*.com

FileName:*.com

FileName

*.cmd

FileName:*.cmd

FileName

*.chm

FileName:*.chm

FileName

*.bat

FileName:*.bat

FileName

*.bas

FileName:*.bas

FileName

*.asx

FileName:*.asx

FileName

*.app

FileName:*.app

FileName

*.adp

FileName:*.adp

FileName

*.ade

FileName:*.ade

Table 1: File Name Extensions and Content Types

You can add additional File Extensions or File Names to this list using the Add-AttachmentFilterEntry CMDlet, if you wanted to filter out zip files, you would need to run the following command: Add-AttachmentFilterEntry -Name *.zip -Type FileName. If you want to filter out messages with a specific MIME type such GIF files, you would need to type: Add-AttachmentFilterEntry -Name image/gif -Type ContentType. If you want to filter out messages that contain an attachment with a specific file name, let’s for example say a file name called dangerous_file, you should type: Add-AttachmentFilterEntry -Name dangerous_file -Type FileName.

If you later on want to remove an attachment filter entry, you do so using the Remove-AttachmentFilterEntry CMDlet. If you for example wanted to remove the zip attachment filter entry, you would need to type: Remove-AttachmentFilterEntry –Identity filename: *.zip.

That is pretty simple right?

In order to be able to use more advanced features such as scanning files in a ZIP file, you would need to install Forefront Security for Exchange Server (which we will talk a bit about later in this chapter) or a supported 3rd party product.

As mentioned you can either choose to block a whole message including the attachment (will return a delivery status notification to the sender), strip the attachment but allow message through (will replace the attachment with a text file explaining why the attachment was stripped) or silently delete both the message as well as the attachment (will delete both without notifying the sender).

You can also configure a custom response message that will be included in the delivery status notification, which is returned to the sender when a message and an attached file are blocked. This is done using the Set-AttachmentFilterListConfig CMDlet. An example could be: Set-AttachmentFilterListConfig –Action Reject -RejectResponse "This message has been rejected since the attached file type isn’t allowed in this organization”.

Note:
All attachment filter entries on an Edge Transport server uses the same attachment filtering behavior, that means same custom response message as well as action (Reject, Strip or SilentDelete).

If you only want to strip the attachment but allow the message through, you would need to type: AttachmentFilterConfigList –Action Strip. If you want to include a custom admin message in the text file that replaces the stripped attachments, you would need to type: AttachmentFilterConfigList –Action Strip –AdminMessage “The attachment in this message has been filtered as it’s not allowed in this organization. Finally in order to silently delete both the message and attachment use: AttachmentFilterConfigList –Action SilentDelete.

The last thing I wanted to mention concerning the Attachment Filtering agent is that you can exclude a list of connectors from attachment filtering, which means that attachment filtering isn’t applied to messages flowing through the specified connectors. You can exclude one or more connectors using Set-AttachmentFilterListConfig –Action Reject –ExceptionConnectors <Connector_GUID>. To get the GUID for a receive connector type: Get-ReceiveConnector | FL.

If you want to see a list of the current settings for AttachmentFilterListConfig, type Get-AttachmentFilterListConfig and hit Enter (see Figure 6).


Figure 6: Attachment Filter List Configuration Settings

For any additional information on how to configure the attachment filtering behavior using the Set-AttachmentFilterListConfig CMDlet, see the Exchange Server 2007 Help file or type Get-Help Set-AttachmentFilterListConfig in the Exchange Management Shell.

Sender Reputation

The Edge Transport server also includes a brand new anti-spam feature called Sender Reputation. The Sender Reputation agent which is enabled by default (although only for externally received messages) is an anti-spam feature that blocks inbound messages according to characteristics of the sender. The agent actually relies on persisted data about the sender, so that it can determine which action to take on inbound messages.

The Sender Reputation agent analyses whether a sender forges the HELO/EHLO statement when establishing an SMTP session to the Edge Transport server. This is done on a per-sender basis which makes it easier to see whether it’s a spammer or a legitimate sender. A spammer typically provides many different unique HELO/EHLO statements in a specific time period, and they often also provide an IP address in the HELO/EHLO statement which doesn’t match their original IP address (that is the IP address from which the connection originates). In addition they often try to provide a local domain name, which is the name of the organization to which the Edge Transport server belongs. In most cases the behavior of a legitimate sender is to use different but a more constant set of domains in the HELO/EHLO statement.

The Sender Reputation agent also performs a reverse DNS lookup when an external SMTP server establishes an SMTP session. This means that the Edge Transport server verifies that the IP address of the SMTP server matches the registered domain name, which the server submits in the HELO/EHLO command. If the IP address does not match the resolved domain name, there’s a good chance we’re dealing with a spammer.

As you already know an inbound message is assigned an SCL rating when the Content Filter is applied. This SCL rating is also persisted for analysis by the Sender Reputation agent. The agent calculates statistics about a sender by looking at how many messages from that sender in the past, had either a low or high SCL rating.

Lastly the Sender Reputation agent is capable of performing an open proxy test against the senders IP address. If the connection is looped back to the Edge Transport server through known open proxy ports and protocols, more specifically SOCKS 4 and 5, Wingate, Telnet, Cisco, HTTP CONNECT and HTTP POST, the sending server is considered an Open Proxy. As you can see in Figure 7 you enabled this feature on the Property page of Sender Reputation.

Note:
In order for the Edge Transport server to perform an Open Proxy text against an external server, keep in mind that you need to open the required outbound ports in any firewall located between the Edge Transport Server and the Internet. The following ports are used during an Open Proxy test: 1080, 1081, 23, 6588, 3128 and 80. If you’re using a proxy server in your organization you also need to configure the Sender Reputation agent to use the proxy server for Open Proxy tests. You do this using the Set-SenderReputationConfig –ProxyServerName CMDlet. For details on how to configure a proxy type Get-Help Set-SenderReputationConfig in an Exchange Management Shell (EMS) or see the Exchange Server 2007 Help File.


Figure 7: Sender Confidence Tab on the Sender Reputation Property Page

Depending on the result of the above mentioned analysis and tests, the Sender Reputation agent assigns a Sender Reputation Level (SRL) to the sender. This SRL can, as is the case with the SCL rating, be a number between 0 and 9. The higher a SRL rating that is assigned to a sender the more likely is it the sender is a spammer. Under the Action tab which also is found on the Sender Reputation property page, you can configure a SRL Block Treshold (see Figure 8), and when the threshold is exceeded the sender is added to the IP Block list for a specified amount of hours (default is 24 hours).


Figure 8: Action Tab on the Sender Reputation Property Page

The Sender Reputation agent does not affect how blocked messages are handled, this is instead controlled by the Sender Filter agent, which can be configured to block, reject or stamp message with blocked sender and continue processing.

Note:
Senders that have not been analyzed by Sender Reputation yet are assigned an SRL rating of 0. Only after the Edge Transport server has received 20 or more messages from a particular sender is an SRL calculated.

Antivirus Scanning

After a given message has been through the Attachment Filter it will be scanned by the antivirus product installed on the server, this could be ForeFront Security for Exchange Server 2007 (which is included in the Exchange 2007 Enterprise CAL) or a supported 3rd party product.

It should come as no surprise the Edge Transport server role integrates perfectly with the ForeFront Security for Exchange Server 2007 product, but the server role also has rich support for partner antivirus providers. So you’re not bound to using the ForeFront Security for Exchange Server product if you choose to deploy an Edge Transport server in your organizations perimeter network (aka DMZ or screened subnet).

Some of the 3rd party products that support the Exchange Server 2007 Edge Transport server role are:

  • Symantec

  • Trend Micro

  • GFI

  • Kaspersky

  • McAfee

  • Sophos

  • Vamsoft

Outlook Junk E-mail Filtering

When a message has been through all the filtering agents, the message will finally be sent to the recipient(s) mailbox, where the Outlook Junk E-mail Filter will take the appropriate action depending on the SCL rating of the message. If the message has an SCL rating which is equal to or greater than the SCL Junk E-mail folder threshold, which is specified on the Content Filtering property page, it will be moved to the Junk E-mail folder in the recipient’s mailbox. Details about the Outlook 2007 Junk E-mail filter are out of the scope of this article, but I can say that the filter has been improved even further since Outlook 2003.

Conclusion

In part 4 and 5, we went through the antispam agents included with the Edge Transport server role. I explained in which order the agents trigger on an inbound message as well as provided miscellaneous tips and tricks in regards to the configuration of the agents and this Exchange 2007 server role in general. Next time, I will give you an insight into how you can deploy a load balanced Edge Transport server setup in your perimeter network. Until then, take care.

If you would like to read the other parts in this article series please go to:

Henrik Walther

Share
Published by
Henrik Walther

Recent Posts

What’s next in the evolution of biometrics and facial recognition technology?

Facial recognition technology has matured to the point of being reliable — for better or for worse. What does the…

1 hour ago

Locking down your Exchange server with cipher suites

Cipher suites are a set of algorithms you need to secure your environment, either by using SSL and TLS. Here’s…

4 hours ago

AI cyber risks: What to look out for when deploying AI technology

Artificial intelligence has greatly improved modern life. But businesses must recognize that AI cyber risks exist and take appropriate measures.

21 hours ago

Review: Office 365 synchronizing and administration tool CiraSync

CiraSync offers an enterprise solution for syncing global address list contacts and calendars to smartphones and other mobile devices. Here’s…

1 day ago

HIPAA IT compliance: Privacy and security rules you must know

HIPAA is the mandatory health regulation that must be followed strictly. But if you’re an IT pro in the health-care…

1 day ago

Exchange in-place upgrade? Sorry, folks, just say no!

An Exchange in-place upgrade would be a dream come true. But if you try it, you will find yourself trapped…

2 days ago