Supporting ISA Server 2000 Publishing of Exchange Server 2000/2003 with SMTP Relays – Part 3: Creating a Simple Anonymous Inbound SMTP Relay and Links to More Resources

In part 1 of this three part series on SMTP relays we talked about the definition and functions of an SMTP relay and how they’re used to protect Exchange Servers protected by an ISA Server firewall. In part 2 we went into more detail and described the features and functions of the various types of SMTP relays used in production networks. Make sure to check out these articles if you haven’t had a chance to do so yet.

Get the Book!

In this article on creating a simple anonymous inbound SMTP relay, the SMTP relay computer is on a machine between the ISA Server firewall and the Exchange Server. So, there are three computers, the ISA Server firewall, the SMTP relay and the Exchange Server. Its important to note that this isn’t the only way you can configure a simple anonymous inbound SMTP relay. In fact, you can find information on how to configure the ISA Server firewall as an anonymous inbound SMTP relay in the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit.

The most common type of SMTP relay is the anonymous inbound SMTP relay. This type of SMTP relay is used to allow inbound SMTP mail to domains under your administrative control. The anonymous inbound SMTP relay accepts mail for your domains and drops mail from domains that you are not responsible for. This prevents spammers from stealing your SMTP server and bandwidth.

In this article we’ll cover the following topics and procedures:

  • Install the IIS SMTP service
  • Configure the IIS SMTP service Properties to prevent open relay
  • Create the Remote Domain
  • Configure the Remote Domain to relay to your Exchange Server
  • Testing the SMTP Server for open relay

Let’s get started!

Install the IIS SMTP Service

For security reasons, IIS 6.0 is not installed by default on a Windows Server 2003 computer. The IIS SMTP service is required on the SMTP relay computer so you must install it if you don’t already have it installed.

Perform the following steps to install the IIS 6.0 SMTP relay computer:

  1. Click Start, point to Control Panel and click on Add or Remove Programs. In the Add or Remove Programs window, click the Add/Remove Windows Components button. You will see a Windows Setup dialog box asking your to please wait.

  1. In the Windows Components dialog box, click on the Application Server entry (do not put a checkmark in its checkbox!). Click Details.

 

  1. In the Application Server dialog box, Click on the Internet Information Services (IIS) entry (do not put a checkmark in its checkbox!). Click Details.

 

  1. In the Internet Information Services (IIS) dialog box, put a checkmark in the SMTP Service checkbox. Click OK.

 

  1. Click OK in the Application Server dialog box.

 

  1. Click Next on the Windows Components dialog box.

 

  1. A progress bar appears as the application installs.

 

  1. Click Finish on the Completing the Windows Components Wizard page.

 

 

Configure the Virtual Non-Authenticating SMTP Server for Anonymous Inbound Relay

 

A virtual SMTP server is created when you install the IIS SMTP service. The first virtual SMTP server is name the Default SMTP Virtual Server. You can create multiple SMTP virtual servers on the same computer. If you create multiple SMTP virtual servers, then you can bind a different IP address to each virtual SMTP server and configure the authentication and relay properties for each server in a different way. This allows you to create authenticating and non-authenticating SMTP virtual servers on the same physical computer.

In this article we’ll focus on the authenticating SMTP relay. You can get information on how to configure authenticating and non-authenticating SMTP virtual servers on the same machine in several of the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit articles. Please check the table of contents for details.

Perform the following steps to safely configure the Default SMTP virtual server:

  1. Click Start and point to Administrative Tools. Click on the Internet Information Services (IIS) Manager entry.

 

  1. Right click on the Default SMTP Virtual Server entry in the left pane of the console. Click the Properties command.

 

  1. In the Default SMTP Virtual Server Properties dialog box, click the General tab. Click the down arrow for the IP address drop down list box and select the IP address that you want the anonymous inbound SMTP relay to use. This anonymous relay accepts mail for the mail domains that are under your administrative control and drops all other inbound mail. Internet SMTP servers use this machine to send mail to the Exchange Server. Click Apply after selecting the IP address.

 

  1. Click on the Messages tab. You have the following options:

Limit message size to (KB)
This is what Microsoft SMTP Service will advertise, in kilobytes, as the maximum message size this SMTP virtual server will accept. If a mail client sends a message that exceeds the limit, the client will get an error. If a remote server supports EHLO, it will detect the advertised maximum message size value when it connects to the SMTP virtual server and won’t even attempt to deliver a message that exceeds the limit. Instead it will simply NDR the sender of the message. A remote server that doesn’t support EHLO will try to send a message that exceeds the size limit, but will still end up sending an NDR to the sender when the message doesn’t go through. The default is 2048 KB. The minimum value is 1KB. To have no limit, clear the check box.

 

Limit session size to (KB)
This is the maximum amount of data, in kilobytes, accepted during the total connection. It is the sum of all messages sent during the connection (applying to the message body only). Type a value larger than the Limit message size to (KB). This maximum should be set carefully, because the connecting message transfer agent (MTA) is likely to resubmit the message repeatedly. The default size is 10240 KB. This value should be greater than or equal to the value entered for Limit message size to (KB). To have no limit, clear the check box.

 

Limit number of message per connection to
When the check box is selected, this option enables you to limit the number of messages sent in a single connection. The default is 20. This feature also provides a method to increase system performance by using multiple connections to deliver messages to a remote domain. Consequently, once the set limit is reached, a new connection is automatically opened and the transmission continues until all messages are delivered. To disable this feature and have no limit, clear the check box.

 

Limit number of recipients per message to
This setting limits the maximum number of recipients for a single message. The default is 100, which is the minimum required number specified in Request for Comments (RFC) 821. To disable this feature and have no limit, clear the check box. Some clients return messages with a non-delivery report (NDR) once an error message is received indicating that the maximum number of recipients has been exceeded. A server running Microsoft SMTP Service does not return messages with an NDR in this instance. It opens a new connection immediately and processes the remaining recipients. For example, if the recipient limit is set to 100 and a message with 105 recipients is being transmitted, the first 100 are delivered in one connection after receipt of the error message. Then a new connection is opened and the message is processed for the remaining five recipients.

 

Send copy of Non-Delivery Report to
When a message is undeliverable, it is returned to the sender with a non-delivery report (NDR). You can designate that copies of the NDR are sent to a specific SMTP mailbox. Type an e-mail address for the mailbox.

 

Badmail Directory
When a message is undeliverable, it is returned to the sender with a non-delivery report (NDR). You can designate that copies of the NDR are sent to a location of your choice.

All NDRs go through the same delivery process as other messages, including attempts to resend the message. If the NDR has reached the retry limit and cannot be delivered to the sender, a copy of the message is placed in the Badmail directory. Messages placed in the Badmail directory cannot be delivered or returned. Check the directory regularly and reconcile the messages, because a full directory may adversely affect Microsoft SMTP Service performance.

If you choose to hold mail evaluated by the SMTP message screener, then you might consider moving the location of the Badmail directory. If you server is subject to a spammer’s spam attack, you want to make sure that the held mail doesn’t fill up the operating system partition.

Make your configuration changes and click Apply.

 

  1. Click on the Delivery tab. Notice the default entries for the retry intervals. If the SMTP relay is not able to contact the Exchange Server, it will attempt to redeliver the mail based on these intervals. Not that after the third failed attempt, the SMTP relay will continue to try and deliver the mail every 240 minutes. You might want to reduce this value in the event that you need to periodically take the Exchange Server offline for maintenance.

  1. Click on the Access tab. You have a number of options available on this tab. Click on the Relay button that’s located in the Relay Restrictions frame (figure 25).

 

  1. The default setting in the Relay Restrictions allows no machines to relay through this virtual SMTP relay except for authenticated users (figure 26). This is a global setting for the SMTP service. We will override this relay configuration by configuring a Remote Domain on this SMTP server later.

We do not want anyone to have “open relay” access to this virtual machine, regardless of their ability to authenticate. Remove the checkmark from the Allow all computers which successfully authenticate to relay, regardless of the list above. Removing this option prevents this virtual server from being able to relay to any mail domain except for those mail domains you create Remote Domain entries for.

Click OK.

 

Note:
Spammers are intensifying their criminal efforts by using brute force and dictionary attacks against an authenticating SMTP relay. You should always remove the checkmark from the Allow all computers which successfully authenticate to relay, regardless of the list above checkbox. If you require an authenticating SMTP relay, you should also require that the authenticating client establish a secure SSL link with the SMTP service before authentication is allowed. Spammers do not have access to your private CA certificate and in most cases they will not even attempt to negotiate a secure SSL link prior to their brute force or dictionary attacks. Be aware that if the ISA Server SMTP filter is enabled, no remote user will be able to negotiate an SSL session with the SMTP service.

  1. Click Apply and then click OK to close the Default SMTP Virtual Server Properties dialog box.

 

You may have noticed that we didn’t make any configuration changes to the authentication mechanism or the relay characteristics of this virtual SMTP server. The reason is that the default setting is to not relay mail unless a user authenticates. This prevents spammers from hijacking your anonymous inbound SMTP relay while allow users to relay if they authenticate. You’ll create a dedicated virtual SMTP server that will act as both a secure and authenticating SMTP relay.

 

Get the New Book!

 

Create the Remote Domain

 

The next step is to configure Remote Domains. You need to create a remote domain for each domain you want to accept inbound mail. For example, if you host the mail domains internal.net and domain.com on your Exchange Server, then you need to create remote domains for both internal.net and domain.com. In the current example we’ll create a single remote domain for internal.net.

 

Perform the following steps to create a Remote Domain that allows anonymous inbound relay:

  1. Open the Internet Information Services (IIS) Manager console and expand your server name . Expand the Default SMTP Virtual Server and right click on the Domains node. Point to the New command and click on Domain.

 

  1. On the Welcome to the New SMTP Domain Wizard page (figure 26), select the Remote option and click Next.

 

  1. On the Domain Name page, type in the name of your mail domain in the Name text box. Click Finish (figure 27).

 

Get the New Book!

 

Configure the Remote Domain to Relay to the Exchange Server

 

The remote domain needs to be configured to relay to the Exchange Server computer. By default, the remote domain does not relay SMTP messages. You configure the remote domain to forward all mail sent to that remote domain to the Exchange Server.

Perform the following steps to configure the remote domain to forward mail to the Exchange Server:

 

  1. Right click on your remote domain in the right pane of the console and click on the Properties command.

 

  1. On the remote domain’s Properties dialog box, put a checkmark in the Allow incoming mail to be relayed to this domain checkbox. This allows the virtual SMTP server to relay mail addressed to this remote domain. Remember, this virtual SMTP server does not relay mail and drops all incoming SMTP messages, with the exception being for users who authenticate and for mail addressed to a domain that you’ve configured a remote mail domain for.

Select the Forward all mail to smart host option and type in a FQDN or IP address for the Exchange Server on your internal network. If you use a FQDN, make sure this SMTP relay computer can resolve this name to the IP address of the Exchange Server’s virtual SMTP server. If you use an IP address, make sure you surround the address with straight brackets, as seen in.

Click on the Security button.

 

  1. By default, this virtual SMTP server does not send credentials to the Exchange Server when it relays mail, and the Exchange Server’s SMTP service does not require credentials. You do have the option of configuring the Exchange Server to require authentication before it will accept the connection from the SMTP relay computer. If you configure the Exchange Server’s SMTP service to require authentication, then you must include valid credentials here. The account and password you enter in this dialog box must match the account you configure on the Exchange Server.

In this example, we will allow anonymous connections to the Exchange Server’s SMTP service. The Anonymous access option is select by default and we will leave it at its default. If you make a change on the Outbound Security dialog box, click OK. Otherwise, click Cancel.

 

  1. Close the Internet Information Services console.

 

Get the New Book!

 

Test Your Server for Open Relay

 

If you have carried out all the procedures described in this article and understood why you carried them out, then your server will not act as an open SMTP relay. However, you should always test your SMTP server to confirm that it was not misconfigured.

There are several methods you can use to test if your machine is an open relay. One that I like to use is a free service provided by the good people at zoneedit.com.

The first step is to open their page at http://www.zoneedit.com/smtp.html. You then type in the following information based on the fields found on the Web page.

 

 

 

Enter the host name or IP address of the email server:
Enter the name or IP address of your SMTP relay in this box. When publishing the SMTP relay using an ISA Server firewall, use the IP address on the external interface of the ISA Server firewall that is used by the SMTP Server Publishing Rule you’re using to publish the SMTP relay

 

Enter From: email address:
Enter an email address from someone in your organization. In fact, it does not matter what email address you use here, as long as this address is not in some way being blocked by your SMTP relay server.

 

Enter To: email address:
Enter an email address for a user in a mail domain not under your administrative control. In this example I entered the address [email protected]. Since the SMTP relay I’m testing is not authoritative for the spam.net domain, it should reject any relay attempt to users in that domain.

 

Begin Test button
Click the button to begin the test!

Notice the output on the bottom of the figure above. The answer my SMTP relay returns to the test machine is 550 5.7.1 Unable to relay for [email protected] indicates that my machine is not configured as an open relay.

 

Get the Book!

 

For More Information on using SMTP Relays to Support ISA Server 2000 Exchange Server Publishing

 

There are several articles in the ISA Server 2000 Exchange Server 2000/2003 Deployment Kit that provide very detailed information on configuring a variety of SMTP relays. The kit provides everything you need to know, including the step by step procedures, to roll out:

 

 

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top