Supporting ISA Server 2000 Publishing of Exchange Server 2000/2003 with SMTP Relays – Part 2: Types of SMTP Relays

Supporting ISA Server 2000 Publishing of Exchange Server 2000/2003 with SMTP Relays
Part 2: Types of SMTP Relays


By Thomas W Shinder M.D.

In part 1 of this series on SMTP relays, we went over what an SMTP is, what it does and why you want one. Head on over to http://www.msexchange.org/articles/smtprelaypart1.html to read part 1 if you haven’t had a chance to look at it yet.

Get the Book!

In this, part 2 of our three part series on SMTP relays, we’ll go over the different types of SMTP relays you can use to protect and enhance your Exchange Server. I’ll explain the different types of SMTP relays you can use for both inbound and outbound access and the advantages provided by each relay type.

Let’s get started!

Types of SMTP Relays

There are several common types of SMTP relays in use on production networks. The types of SMTP relays we’ll talk about in this article include the following:

  • Open SMTP Relay
  • Anonymous SMTP Relay
  • Authenticating SMTP Relay
  • Filtering SMTP Relay
  • Secure SMTP Relay
  • Inbound SMTP Relay
  • Outbound SMTP Relay
  • Keep in mind when you’re reading about the various SMTP relays that these types are not “mutually exclusive”. Any SMTP relay computer can act as multiple SMTP relay types. I’m listing them separately so that it will be easily to understand how each type of SMTP relay operates.

    Open SMTP Relay

    The open SMTP relay is what the RBL’ers live for. They get their kicks when they can blacklist a poor network admin who forgot to prevent his machine from being an open relay. The RBL’er puts the address of the open relay in his list and you might as well try to fly to the moon in a go-cart before you get the dreaded RBL’er to remove you from his list.

    An open SMTP relay is an SMTP server that accepts anonymous connections from any SMTP client or server in the world and relay SMTP messages to any SMTP server in the world. Depending on the speed of the connection the spammer and the SMTP relay computer are using, an evil spammer can send millions of spam messages via an open SMTP relay in a short period of time,.

    Figure 3

    shows how a spammer can hijack an open relay to send his vile missives to any user in the world. The spammer has a laptop computer containing his fraudulent and pornographic spam messages. The laptop also has a database of millions of email addresses for thousands of mail domains. The spammer has installed a wireless NIC in his laptop and has gone to various Web sites where so-called “war chalkers” have provided locations of open wireless “hot spots” that do not require authentication to access the Internet. These helpful war chalkers make it easy for the contemptible spammer to hijack a wireless Internet connection and steal bandwidth from an unsuspecting business.

    Note:


    You can bet that some poor businessman will wonder why he can no longer send mail to his customers. The most likely reason will be that the other side of the spammer coin, the RBL’er, has joyfully placed the poor victim’s address in his dreaded RBL.

    The spammer uses the SMTP client application on his laptop to pump his junk to users all over the Internet when he has has identified your SMTP server as an open relay. How did he find out you had an open relay? The RBL’er told him. He configures his SMTP client application to use your SMTP server as its outbound mail SMTP server.

    In this example the open relay is an SMTP server published by an ISA Server firewall. The ISA Server firewall confirms the SMTP commands sent by the spammer are valid and that none of them represent a buffer overflow condition. Then the ISA Server firewall passes the SMTP messages to the published SMTP relay on the internal network. The SMTP relay is configured as an open relay, so it does not require authentication and it is able to relay messages to all domains in the world.

    When the open SMTP relay receives the spam, it performs a DNS query for the destination domains. Suppose the SMTP relay received spam messages for [email protected], [email protected] and [email protected]. The SMTP relay computer must query a DNS server for the MX and A record information for the mail.com, stuff.com and domain.com domains. After the SMTP relay identifies the SMTP servers responsible for mail to each of these domains, it sends the mail.

    Figure 3

    Its clear that you do not want your SMTP relay computer to be an open SMTP relay. You’ll end up on some RBL’ers blacklist and the spammer will consume all your available bandwidth with his despicable trash. I’ll show you how to configure your IIS SMTP Server to not be an open relay in part three of this article.

    Anonymous SMTP Relay

    An anonymous SMTP relay is quite different form an open SMTP relay. You need to configure at least one anonymous SMTP relay if you want to receive mail for domains under your administrative control,.

    Figure 4

    shows how an anonymous SMTP relay works. Suppose an employee on a remote network sends mail to someone at [email protected]. The employee’s laptop is configured with Outlook Express as an SMTP/POP3 client. The user sends his message to [email protected] and the message goes to an Internet SMTP server.

    The Internet SMTP server is not responsible for mail destined to internal.net so it does the required DNS queries to find the name of the server and the IP address of the server responsible for internal.net mail. The DNS queries result in the server name mail.internal.net and the IP address 2.2.2.2. The Internet SMTP server relays the message to [email protected] to this address.

    This address is the address on the external interface of the ISA Server firewall that is used in the SMTP Server Publishing Rule publishing the SMTP relay on the internal network. The ISA Server firewall forwards the SMTP messages to the SMTP relay on the internal network.

    The SMTP relay computer uses remote domains to determine if it should accept or reject the incoming mail. The SMTP relay computer is configured with two remote domains and is also configured to prevent open relay. The SMTP relay in this example is configured with the following remote domains:

    internal.net

    company.com

    Both of these remote domains are configured to relay mail to the Exchange Server on the internal network. The Exchange Server is configured with a recipient policy that makes the server an authoritative endpoint for SMTP messages for internal.net and company.com SMTP mail.

    The SMTP relay is also configured to reject mail destined for any other domain. In contrast to the open relay, the anonymous SMTP relay is not able to relay mail to all domains – the anonymous SMTP relay can only relay mail to domains listed in its remote domains list and only mail destined for those domains is relayed to the Exchange Server.

    We call this an anonymous SMTP relay because no authentication is required to connect to the SMTP relay computer. It’s important that authentication not be required, because Internet SMTP servers will not be able to authenticate. You do not want to require authentication from SMTP servers and clients on the Internet when the mail they send is destined for domains under your administrative control.

    Figure 4

    The anonymous SMTP relay is the most common type of inbound relay server. You will always configure an anonymous SMTP relay to allow inbound access for mail destined to the domains you control. In the example above, we are responsible for the internal.net and company.com domains and so no server or client should ever have to authenticate to send mail destined for those two domains.

    Get the New Book!

    Authenticating SMTP Relay

    The authenticating SMTP relay requires users to authenticate before the SMTP relay computer accepts the connection request. Authenticating SMTP relays are not useful for allowing inbound SMTP messages to domains that are under your administrative control. However, the authenticating SMTP relay is quite helpful if you have remote users who require an SMTP server to send outbound SMTP messages too.

    Figure 5

    describes the setup for an authenticating SMTP relay. Suppose you have remote users who stay at various hotels all over the world. The users connect their laptops to the hotel’s broadband network. The hotel provides them with a connection to the Internet, but it does not provide them with an SMTP server address. It’s up to the user to find his own SMTP server to send outbound SMTP messages to. This issue is becoming increasingly problematic as more people stay at facilities that provide broadband access but no SMTP server address.

    This problem is solved by you providing an SMTP server for your remote users. Requirements for such an SMTP include:

  • The server is an SMTP relay computer that can relay to all mail domains, including the domains that you have administrative control over
  • The server is published via the ISA Server firewall so that the ISA firewall’s SMTP filter can protect it against SMTP exploits
  • The server must require authentication before a connect attempt to it is allowed
  • The server must allow open relay to all email domains
  • In figure 5 you can see the remote employee using a laptop to send mail from his email client application. In this example the remote employee uses Outlook Express and he has configured his laptop to send SMTP mail to the SMTP server you have created for him, which is located on the internal network behind is the ISA Server firewall.

    The user sends a message to [email protected]. Since there is no local SMTP server, the message is sent directly to the IP address on the external interface of the ISA Server firewall that is being used by the SMTP Server Publishing Rule to publish the SMTP relay on the internal network. The ISA Server firewall forwards the SMTP connection request to the SMTP relay computer. The SMTP relay computer challenges the mail client application for credentials, Outlook Express supplies the required credentials and successfully logs on to the SMTP relay computer.

    The SMTP relay is configured to allow open relay to all email domains when the user successfully authenticates. This allows the authenticated user to send mail to all email domains, including domains under your administrative control. In this example, you have administrative control over internal.net and company.com. The message is destined to the stuff.com domain, which is not under your administrative control, so the SMTP relay computer will need to relay the SMTP message addressed to [email protected] to an SMTP server that is responsible for the stuff.com domain.

    The SMTP relay performs DNS queries and finds that the MX record for the stuff.com domain. This record indicates that mail.stuff.com is responsible for stuff.com mail and that the IP address of mail.stuff.com is 3.3.3.3. The SMTP relay on your internal network relays the message to [email protected] to the this address.

    The defining concepts behind the authenticating SMTP relay is that it allows open relay only to authenticated users. Unauthenticated users cannot relay to domains not under your administrative control. This prevents spammers from using your SMTP relay to send mail to domains outside your control. Of course, spam destined for your domains will still make it to your users’ mailboxes. You have to use other methods to block spam destined for you own domains.

    Figure 5


    Note:


    Criminal spammers are upping the ante on their nefarious deeds. These Internet criminals are now trying to subvert security provided by authenticating SMTP relays by performing dictionary or brute force attacks against the authenticating SMTP relay. If the spammer successfully guesses a valid username and password, the noisome spammer will be able to use the SMTP relay computer as an open relay. Therefore, in spite of the fact that your SMTP relay computer is not configured as an open relay, the slavering RBL’ers will be quick to insert your SMTP server’s address into their RBL lists.

    Get the New Book!

    Filtering SMTP Relay

    A filtering SMTP relay helps control spam and other malicious content. One of the greatest risks to corporate networks today comes from the content received in user mailboxes. The problem can be the nature of the message itself (pornography or corporate secrets) or the nature of the attachment (viruses and Trojans contained in executables). A filtering SMTP relay can examine the content of the SMTP messages and delete or quarantine those of questionable nature.

    Figure 6

    shows how a filtering SMTP relay works. Mail for your organization is received from Internet SMTP servers from all over the world. Your organization’s mail arrives at the external interface of the ISA Server firewall and the SMTP Server Publishing Rule on the firewall forwards inbound SMTP messages to the SMTP relay on the internal network. In this example, the SMTP relay is using the ISA Server 2000 SMTP Message Screener to filter SMTP mail.

    The ISA Server SMTP Message Screener can be installed on the ISA Server firewall, a dedicated SMTP relay computer or the Exchange Server. In this example the SMTP Message Screener is installed on the dedicated SMTP relay. The mail is inspected and SMTP Message Screener applies SMTP Message Screener Policy. You can control inbound SMTP messages using:

  • Keywords
  • Attachments
  • Source email address or source email domain
  • After the SMTP relay has filtered the mail, it will do one of the following:

  • Delete the offending messages
  • Hold the offending messages in the BADMAIL folder on the SMTP relay computer
  • Mail the offending messages to an account you specify
  • All mail that passes inspection is relayed to the Exchange Server on the internal network.

    Figure 6

    A filtering SMTP relay is mandatory in today’s world of overwhelming spam and malicious email payload. You can use the built-in SMTP Message Screener, or you can use third party products such as GFI’s MailEssentials or Sunbelt’s iHateSpam Gateway Edition.

    Get the New Book!

    Secure SMTP Relay

    The secure SMTP relay insures that all SMTP messages sent from the SMTP client on the remote network to the SMTP relay behind the ISA Server firewall are encrypted. The email client application is configured to use SSL encryption when connecting to the SMTP server. The SMTP relay is configured to require the SMTP client application to successfully negotiate a secure link before messages can be sent.

    Figure 7

    describes the connection between the remote mail client and the SMTP relay. The remote user creates an email message addressed to [email protected] and clicks the Send button. The email client sends a connection request to the IP address on the external interface of the ISA Server firewall publishing the secure SMTP relay on the internal network. The ISA Server firewall forwards the connection request to the secure SMTP relay on the internal network. The SMTP service on the SMTP relay computer requests that the client negotiate a secure link. A CA certificate from the CA issuing the SMTP service’s Web site certificate is installed on the client. This allows the client to trust the certificate presented by the SMTP service during the SSL link negotiation. The user’s credentials are sent after the secure SSL link is established and then the SMTP mail is forwarded to the secure SMTP relay after the credentials are accepted.

    Note:


    You do not need to force authentication to use a secure connection. You can configure the SMTP relay to accept an anonymous connection after the secure link is established.

    The secure SSL link allows the remote user to send SMTP messages which often have confidential content to your SMTP relay and not worry about an intruder somewhere in the path intercepting and reading message content. Remote users often connect to hotel Ethernet or wireless networks that allow malicious users to run “packet sniffing” software. These packet sniffers pick up information on the wire and can allow an intruder to obtain not only information on the message content, but the user’s credentials as well.

    Figure 7

    Inbound and Outbound SMTP Relays

    SMTP relays can be configured to support inbound SMTP relay (as seen in the all the examples so far) or outbound SMTP relay. Inbound SMTP relay servers accept incoming SMTP messages from remote SMTP clients and servers. The incoming messages move through the SMTP relay and are either relayed to an external SMTP server (if the destination of the mail message is not hosted by your organization), relayed to your Exchange Server, or dropped.

    Outbound SMTP relay is when the SMTP messages originating on the internal network are first sent to the SMTP relay computer for routing and/or filtering. The outbound SMTP relay accepts SMTP mail from the Exchange Server and then relays that mail to the Internet SMTP server responsible for that mail. The advantage of using an outbound SMTP relay is that the Exchange Server does not need to resolve the MX domain name for the destination of each email message and the Exchange Server does not need to contact Internet SMTP servers. The SMTP relay computer does the hard work of name resolution and the dangerous work of connecting to untrusted SMTP servers.

    Note:


    A smart host is an outbound SMTP relay

    Figure 8

    shows both the inbound and outbound paths. The green arrows demonstrate the inbound path. The external user creates a message addressed to [email protected]. The email client application is configured to send SMTP messages to the IP address of the external interface of the ISA Server firewall that is being used by the SMTP Server Publishing Rule. The message to [email protected] is accepted by the ISA Server firewall and forwarded to the SMTP relay on the internal network. The SMTP relay checks the destination address and determines that its for internal.net, which is a domain under your administrative control. The SMTP relay then relays the messages to the Exchange Server.

    The red arrows demonstrate how outbound SMTP relay works. The Outlook 2003 client on the internal network uses a MAPI connection to connect to the Exchange Server. The Outlook 2003 user creates a message that is addressed for [email protected]. The Exchange Server knows that it is not responsible for the stuff.com mail and forwards the message (in fact, in this scenario, the Exchange Server relays the message) to the outbound SMTP relay on the internal network. The outbound SMTP relay knows that it is not responsible for stuff.com mail and resolves the MX and host A records for the stuff.com domain. The outbound SMTP relay then sends the message to the IP address of the stuff.com SMTP server.

    Figure 8

    It is extremely important for you to realize that even though I have described a number of SMTP relay “types”, these types are not mutually exclusive and its best practice to use them in combination. Some examples of combining the SMTP relay types include:

  • Secure, authenticating open SMTP relay
  • Anonymous inbound relay and secure authenticating inbound SMTP relay
  • Secure, authenticating inbound SMTP relay, anonymous inbound relay and anonymous outbound relay
  • Secure, authenticating inbound SMTP relay, anonymous inbound SMTP relay, authenticated outbound relay
  • Secure, authenticating inbound SMTP relay, anonymous inbound SMTP relay, secure authenticated outbound relay, and SMTP filtering relay
  • You can create any mix you like of the SMTP relay types mentioned in this section. The configuration requirements become increasingly complex as you mix the types, but you can create any mix of these types using the IIS 5 or IIS 6 SMTP service. Several of these configurations are described in exact, step by step detail in the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit. If you’re interested in putting together an SMTP relay solution and need to know exactly how to configure it, then you’ll always be a winner with the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit.

    In the third and last part of this series on SMTP relay you’ll see how to create a simple anonymous inbound SMTP relay for domains under your administrative control. I’ll also discuss some important resources for your that provide detailed step by step instructions on how to create and configure the more sophisticated SMTP types. I’ll also show you the actual configuration interfaces that allow you to apply all the principles you learned in part 1 and part 2 (this article) of this series. See you then!

    Get the Book!

    I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=6;t=002096 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom

    If you would like us to email you when Tom Shinder releases another article on ISAserver.org, subscribe to our ‘Real-Time Article Update’

    by clicking here. Please note that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy!

    Leave a Comment

    Your email address will not be published.

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

    Scroll to Top