Configuring an Inbound and Outbound SMTP Relay to Complement ISA Server 2004 Firewall Protection for Exchange Servers, Part 2: Step by Step Instructions Including MailEssentials 9

Configuring an Inbound and Outbound SMTP Relay to Complement ISA Server 2004 Firewall
Protection for Exchange Servers, Part 2: Step by Step Instructions Including MailEssentials 9



By Thomas W Shinder M.D.

In part 1 of this two part article on how to create an inbound and outbound SMTP relay to protect your Microsoft Exchange Servers we discussed the principles of SMTP relay and how relay can protect your Exchange Servers from the risks of direct contact with Internet SMTP and DNS servers. If you missed that article, you can check it out at http://www.isaserver.org/articles/smtprelayinboundoutbound.html.

Get the New Book!

In this, part 2 of the series, we’ll provide the detailed step by step procedures you need to actually make the theory of secure SMTP relay into reality. First, lets take a look at our simple example network. The figure below provides the details.

The SMTP relay in this example has GFI MailEssentials installed on it. MailEssentials is an excellent anti-spam solution that uses a number of anti-spam technologies to keep productivity sapping and potentially dangerous spam from entering your users’ mailboxes. We will go through the procedures required to install and configure MailEssentials and then test its functionality.

If you do not have the resources to purchase a high-powered anti-spam solution such as MailEssentials, you could use the built-in SMTP Message Screener that is included with your ISA Server 2004 license. The SMTP Message Screener comes with ISA Server 2004 and it can act as a rudimentary spam whacker. However, for full fledged spam blocking and attachment filtering, you will need a dedicated application to supplement the SMTP Message Screener.

External users need to use the SMTP relay to relay mail to domains not hosted by the organization, so we will make the SMTP relay machine a member of the same Active Directory domain that the Exchange Server belongs to. This is required so that the SMTP relay has access to the user account database in the Active Directory. However, you could mirror the Active Directory user database (which can be prohibitive in terms administrative overhead) on the SMTP relay machine or you could create a “group account” and have everyone use the same account to relay to external domains.

In addition, we will request a certificate for the SMTP service on the SMTP relay so that external users can use a SSL/TLS secured connection between your external corporate clients and the SMTP relay. This keeps the data in the outgoing SMTP communications from your remote users from being intercepted in transit. The main reason for this is that remote users tend to connect from shared networks via a hotel or conference center broadband connection, where it is easy to use Network Monitor or Ethereal (or any other network analyzer) to intercept user names and passwords. The SSL/TLS encryption will secure the user credentials.

In order to simplify the process of certificate assignment, we have installed an enterprise CA on the Exchange Server, which is also a domain controller in this example. Using an enterprise CA allows us to use the Certificate Request Wizard integrated into the IIS SMTP service. If we had installed a standalone CA, we would have to perform an offline request, which involves using a text file and submitting the text file via the Web enrollment site.

We will walk through the following procedures:

  • Installing the SMTP service on the SMTP relay
  • Requesting a certificate for the SMTP service on the SMTP relay – this is used to allow secure connections for authenticated users
  • Configuring the SMTP service for inbound secure/authenticated and anonymous relay
  • Installing GFI MailEssentials on the SMTP relay
  • Configuring the Exchange Server to use the SMTP relay for outbound relay
  • Configuring the ISA Server 2004 Firewall to allow the Internal DNS server outbound access to the DNS (TCP and UDP) protocols
  • Configuring the ISA Server 2004 firewall to allow the SMTP relay outbound access to the SMTP protocol
  • Creating an SMTP Server Publishing Rule that allows inbound connections to the SMTP relay on the Internal network
  • Configuring Public DNS entries to support the SMTP Server Publishing Rule
  • Configuring authoritative DNS server entries with your domain Registrar – this is only required if you decide to host your own DNS server
  • Install the root CA certificate on the SMTP client computer
  • Test the connection
  • Installing the SMTP Service on the SMTP Relay

    The first step is to install the SMTP service on the Windows Server 2003 machine acting as the SMTP relay. IIS services are not installed by default on Windows Server 2003 machine, so we must install the service now.

    Perform the following steps to install the IIS 6 SMTP service:

    1. Click Start and click Control Panel. Click Add or Remove programs.
    2. In the Add or Remove programs applet, click the Add/Remove Windows Components button.
    3. On the Windows Components page, select the Application Server entry and click Details.
    4. In the Application Server dialog box, select the Internet Information Services (IIS) entry and click Details.
    5. In the Internet Information Services dialog box, put a checkmark in the SMTP Service checkbox and click OK.
    6. Click OK in the Application Server dialog box.
    7. Click Next on the Windows Components page.
    8. Click OK in the Insert Disk dialog box. In the Files Needed dialog box, enter the path to the i386 folder in the Copy files from text box and click OK.
    9. Click Finish on the Completing the Windows Components Wizard page.

    Requesting a certificate for the SMTP service on the SMTP Relay

    We want remote users to have the ability to secure their connections to the SMTP relay using SSL/TLS encryption. This will protect both the data and the user credentials when remote users connect to the SMTP relay. The first step to making this happen is to request and bind a certificate to the SMTP virtual server.

    Note that I have kept the scenario relatively simple by binding only a single IP address to the SMTP relay. Because we have only a single IP address bound to the SMTP relay, and because we have only a single IP address bound to the external interface of the ISA Server 2004 firewall machine, we will not be able to force SSL connections from the remote users. We will need to depend on the users properly configuring their e-mail client applications to force the SSL/TLS connection.

    The alternative to this configuration is to bind two IP addresses to the external interface of the ISA Server 2004 firewall and two IP addresses to the SMTP relay machine. When there are two IP addresses available on the SMTP relay, we can create a second virtual SMTP server and bind the second address to the second server. Then we could force only SSL/TLS secured connections to the second SMTP virtual server. This is the preferred method of allowing secure SMTP relay because you don’t have to depend on your users to correctly configure their client applications. The second IP address on the external interface of the ISA Server 2004 firewall is required to publish the second virtual SMTP server.

    However, because most of our users on www.isaserver.org have a single public address, I have chosen to keep the configuration simple.

    If you are interested in a detailed description on how to perform the more complex configuration, please let me know and I’ll do an article on that configuration as well. Send me a note at [email protected] and if enough people are interested in this configuration, I’ll do another article demonstrating how to implement the dual IP address solution.

    Perform the following steps to request the certificate for the SMTP virtual server:

    1. Click Start and point to Administrative Tools. Click Internet Information Services (IIS) Manager.
    2. In the Internet Information Services (IIS) Manager console, expand your server name and then expand the Default SMTP Virtual Server entry. Right click the Default SMTP Virtual Server and click Properties.
    3. In the Default SMTP Virtual Server Properties dialog box, click the Access tab. On the Access tab, click the Certificate button in the Secure communication frame.
    4. Click Next on the Welcome to the Web Server Certificate Wizard page.
    5. On the Server Certificate page, select the Create a new certificate option and click Next.
    6. On the Delayed or Immediate Request page, select the Send the request immediately to an online certificate authority option and click Next.

    1. On the Name and Security Settings page, you can accept the default name and Bit length. Click Next.
    2. On the Organization Information page, enter your Organization and Organizational Unit information. Click Next.
    3. On the Your Site’s Common Name page, enter a name that external users will use in their mail client application to reach the SMTP server. This name must resolve to the IP address on the external interface of the ISA Server 2004 firewall that you will use in your SMTP server publishing rule. For example, we will enter mail.msfirewall.org in the Common name text box. The external user will be able to use a public DNS server to resolve this name to the IP address on the external interface of the ISA Server 2004 firewall. It is critical that this name be the same as the name used when configuring the e-mail client application. You will need to enter this name into your public DNS server. We will go over the procedures for configuring the e-mail client application later in this article. Enter the FQDN in the Common name text box and click Next.

    1. On the Geographical Information page, enter your State/Province and City/locality information and click Next.
    2. On the Choose a Certification Authority page, accept the default selection for your enterprise CA. If you have multiple enterprise CA’s, you could select an alternate CA. Click Next.
    3. Review the information on the Certificate Request Submission page and click Next.
    4. Click Finish on the Completing the Web Server Certificate Wizard page.
    5. Click OK on the Default SMTP Virtual Server Properties dialog box.

    Configuring the SMTP Service for Inbound Secure/Authenticated and Anonymous Relay

    Now we’re ready for some heavy lifting. We need to configure the SMTP service to accept incoming mail to our Exchange Server and also configure the SMTP service to relay mail to external domains for authenticated users.

    The first step is to configure a remote domain for the domain that we host. In this example, our public e-mail domain is msfirewall.org. Mail addressed to msfirewall.org should by relayed to our Exchange Server organization. Mail send to any other domain should not be sent to the Exchange Server. This prevents spammers from using our Exchange Server as a spam relay.

    Perform the following steps to configure the remote domain for inbound relay to your organizations Exchange Server:

    1. In the Internet Information Services (IIS) Manager expand the Default SMTP Virtual Server and right click on the Domains node. Point to New and click Domain.
    2. On the Welcome to the New SMTP Domain Wizard page, select the Remote option and click Next.
    3. On the Domain Name page, enter the mail domain your Exchange Server hosts in the Name text box. In this example, we’ll enter msfirewall.org. Click Finish after entering the domain name.

    1. You will see the new remote domain appear in the right pane of the console. Double click on the new remote domain.
    2. In the remote domain’s Properties dialog box, put a checkmark in the Allow incoming mail to be relayed to this domain checkbox. In the Route domain frame, select the Forward all mail to smart host option and then enter the IP address of the Exchange Server that should accept the incoming mail. In this example, the IP address is [10.0.0.2]. Note that you must include straight brackets around the IP address so that the SMTP service does not considered this to be a fully qualified domain name and try to resolve it. Note that you also have the option to use Use DNS to route to this domain. If you use this option, the SMTP relay will use DNS to resolve the name of the remote domain by doing a DNS query for the MX record for the domain. Click Apply and then click OK.

    Creating the remote domain is only the first step in configuring our SMTP relay computer. The next step is to configure the properties of the Default SMTP Virtual Server. Perform the following steps to configure the SMTP virtual server:

    1. In the Internet Information Services (IIS) Manager console, expand the server name and then right click on the Default SMTP Virtual Server entry. Click Properties.
    2. On the Default SMTP Virtual Server Properties dialog box, select the IP address of the SMTP Virtual Server from the IP Address list. Put a checkmark in the Enable logging checkbox.
    3. Click on the Access tab. On the Access tab, click the Authentication button in the Access control frame.
    4. In the Authentication dialog box, leave the checkmark in the Anonymous access checkbox in place. Place a checkmark in the Integrated Windows Authentication checkbox. Click OK.

    1. In the Default SMTP Virtual Server Properties dialog box, click the Relay button in the Relay Restrictions frame.
    2. In the Relay Restrictions dialog box, confirm that there is a checkmark in the Allow all computer which successfully authenticate to relay, regardless of the list above checkbox. This setting will allow your external users to relay through the SMTP relay machine to domains that you do not host. Click the Add button.
    3. In the Computer dialog box, enter the IP address of the Exchange Server in the Single computer text box. We need to allow the Exchange Server to relay mail outbound to external mail domains and this setting will allow the Exchange Server that access. Note that if this Exchange Server had a logged on user, that user’s credentials could be used for outbound relay. However, since servers typically do not have logged on users, you need to enable relay access control by using the Exchange Server’s IP address. This does have the potential for abuse, as spammers could spoof their IP address. However, the ISA Server 2004 firewall rejects spoofed communications, so you are protected from this exploit when using an ISA Server 2004 firewall to protect your organization. Click OK.

    1. In the Relay Restrictions dialog box, you will now see the IP address of the Exchange Server in the Computers list and the access is set as Granted. Click OK.
    2. In the Default SMTP Virtual Server dialog box, click the Delivery tab. On the Delivery tab, click the Advanced button.
    3. In the Advanced Delivery dialog box, you have the option to enter the IP address for FQDN for a smart host. A smart host is an SMTP server that you can forward e-mail messages to and in turn, the smart host will perform name resolution for the destination mail domains for the outgoing e-mail messages. The smart host offloads name resolution responsibility from the Exchange Server or SMTP relay to the smart host computer. You also have the option to allow the SMTP relay to attempt delivery by resolving the SMTP domain name itself before sending the message to the smart host. Only if the SMTP relay is not able to resolve the name itself will it forward the messages to the smart host. Your smart host can be your ISP’s SMTP server, or any other SMTP server that you consider highly effective and available. In this example, we will not enter a smart host address. This will allow the SMTP relay to resolve MX domain name itself. It will also require that we allow our Internal DNS server (located on the domain controller) to perform name resolution for Internet host names. The SMTP relay is configured to use the Internal DNS server, so it will send MX domain name queries to the Internal DNS server, the Internal DNS server will query Internet DNS servers to resolve the MX domain names, and then the Internal DNS server will forward the result to the SMTP relay. Then the SMTP relay will send the e-mail messages directly to the SMTP server responsible for that domain’s e-mail. Click OK.

    1. Click OK in the Default SMTP Virtual Server Properties dialog box.

    Get the New Book!

       

    Installing GFI MailEssentials on the SMTP Relay

    Now we’re ready to install GFI MailEssentials. MailEssentials is easily to set up after you’ve done the pre-installation footwork that we’ve already done so far. You can download the MailEssentials software from www.gfi.com and get an evaluation version if you don’t already own the software.

    Perform the following steps to install the MailEssentials software:

    1. Download the installation file to the desktop and double click on it. Click Next on the Information page.
    2. In the WinAce SFX – GFI Bayesian Analysis Wizard dialog box, change the path to C:\GFIINSTALL. Click Extract.
    3. Click Yes on the Confirmation dialog box asking if you want to launch the setup file.
    4. On the Language page, select the appropriate language and click OK.
    5. Click Next on the Welcome to the GFI MailEssentials Installation Wizard page.
    6. On the Check for latest build availability page, select the Do not check for a new build. Since you just downloaded the file, there should not be a newer version. Click Next.
    7. Select the I accept the license agreement option and click Next.
    8. On the Destination Folder page, select the default location for the MailEssentials files, or install it to an alternate location using the Browse button. Click Next.
    9. On the User Information page, enter your Full Name, Company and License Key if you have one. Otherwise, you can leave the key as Evaluation. Click Next.
    10. On the Mail Server page, enter the IP address of the Exchange Server in the IP Address text box. Leave the on port value as 25. In the Local Domain text box, enter the domain name that you host. Note that MailEssentials concept of local domain is a bit different than IIS 6.0’s concept. In this example, the MailEssentials local domain is msfirewall.org. This is in spite of the fact that the msfirewall.org is configured as a remote domain in the IIS SMTP service’s console. Click Next.

    1. On the Administrator E-mail page, enter the address of a mail administrator who will receive messages from the MailEssentials software. This address is used to send alerts and other information from the MailEssentials software. In this example, we’ll enter [email protected]. Click Next.
    2. On the Active Directory page, you can select whether or not you want to use the Active Directory integration feature that supports custom rules based on user. If you do not have an Active Directory domain, or if you do not want to use Active Directory or if you want to put the MailEssentials software into a DMZ, then you can select the No… option. We will select that option because we want the same policies to apply to all users. However, if you want to support custom policies based on user account, you should use Active Directory. Otherwise, you’ll have to use specific e-mail addresses that you configure into the MailEssentials software. In this example, we will select the No… option. Click Next.
    3. On the Ready to Install the Application page, confirm that you Internal network domains that you host are included in the list. If they are not, you can configure MailEssentials with additional hosted domains after installation is complete. Click Next.
    4. In the SMTP Service dialog box, click Yes to restart the IIS SMTP service.
    5. Click Finish on the GFI MailEssentials has been successfully installed page.

    MailEssentials has a powerful Bayesian filter and pre-populates the filter with over 13,000 sample spam messages. However, you must first train the filter with several thousand non-spam messages (which they refer to as “HAM”). There is a Bayesian filter Wizard that helps you get this feature up and running quickly either by analyzing outgoing mail, or by connecting to a folder in an existing account and analyzing that mail. You can refer to the user manual for details of this configuration. In order to help us quickly demonstrate that our SMTP relay is working, we’ll use the built-in keyword rules that block mail with those keywords.

    Perform the following steps to see what the default keywords are and how to configure the disposition of block e-mail messages:

    1. Click Start and point to All Programs. Point to GFI MailEssentials and then click MailEssentials Configuration.
    2. In the GFI MailEssentials Configuration console, expand the GFI MailEssentials node and then click the Keyword Checking node.
    3. In the right pane of the console, click the Properties link.
    4. In the Keyword Checking Properties dialog box, confirm that there is a checkmark in the Scan e-mail body for the following keywords or combinations of keywords checkbox. Make a note of the blocked keywords. You can keep them, remove some of them or remove all of them. You can also add your own keywords.
    5. Click on the Subject tab. You can do the same with entries in the e-mail message’s subject line. You can add your own, remove some of the default entries or remove all of these entries.
    6. Click the Actions tab. Select the Block e-mail and perform this action option. Select the Move to the specified folder option. In this example, enter C:\ in the text box. This is where the spam messages will be placed. Note that I’m selecting this location for simplicity in this example. You may want to allow the messages to go to the users with the [SPAM] entry added to the message subject. This will allow users to create their own rules to filter spam and also view the spam to make sure that no false positives took place.
    7. Click Apply and then click OK in the Keyword Checking Properties dialog box.

       

    Configuring the Exchange Server to use the SMTP Relay for Outbound Relay

    The Exchange Server will use the SMTP relay for outbound relay via a smart host setting on the Exchange Server’s SMTP service. Perform the following steps to configure the SMTP service on the Exchange Server to use the SMTP relay as its smart host:

    1. On the Exchange Server machine, click Start and point to All Programs. Point to Microsoft Exchange and click on System Manager.
    2. In the Exchange System Manager console, expand the organization name and then expand the Servers node in the left pane of the console. Expand the Protocols node and then expand the SMTP protocol.
    3. Right click the Default SMTP Virtual Server entry in the left pane and click Properties.
    4. In the Default SMTP Virtual Server Properties dialog box, click the Delivery tab. On the Delivery tab, click the Advanced button.
    5. In the Advanced Delivery dialog box, enter the IP address of the SMTP relay computer in the Smart host text box. In this example, the IP address of the SMTP relay is 10.0.0.3, so we will enter [10.0.0.3] into the text box. Notice that straight brackets must be placed around the IP address. Click OK.
    6. Click Apply and then click OK in the Default SMTP Virtual Server Properties dialog box.

    Configuring the ISA Server 2004 Firewall to allow the Internal DNS Server Outbound Access to the DNS (TCP and UDP) Protocols

    The Internal network DNS server must be able to resolve the mail domain name in the outgoing messages to the SMTP server responsible for the e-mail message. In our current scenario, the Internal network DNS server is on the domain controller/Exchange Server machine. This DNS server is configured to resolve Internet domain names.

    By default, the Microsoft DNS server can resolve Internet domain names if you configure the DNS properly before installing the Active Directory. The means you need to install and configure the DNS before you promote the machine to a domain controller. However, that is another subject for another time. The key factor here is that the DNS server must be primed with the proper root hints file that allows it to perform recursion to resolve Internet host names.

    In order for the Internal network DNS server to resolve Internet host names, it must be able to perform recursion. This requires that the DNS server be able to access DNS servers on the Internet. We must create an Access Rule on the ISA Server 2004 firewall to allow the Internal network DNS sever to resolve Internet host names.

    Perform the following steps to create the DNS Access Rule:

    1. At the ISA Server 2004 firewall computer, open the Microsoft Internet Security and Acceleration Server 2004 management console and expand the server name. Click the Firewall Policy node.
    2. Right click the Firewall Policy node, point to New and click Access Rule.
    3. On the Welcome to the New Access Rule Wizard page, enter a name for the rule in the Access Rule name text box. In this example we’ll call it Outbound DNS. Click Next.
    4. On the Rule Action page, select Allow and click Next.
    5. On the Protocols page, select the Selected protocols option from the This rule applies to list. Click the Add button.
    6. In the Add Protocols dialog box, click the Common Protocols folder and double click on DNS. Click Close.
    7. Click Next on the Protocols page.
    8. On the Access Rule Sources page, click the Add button.
    9. In the Add Network Entities dialog box, click the Networks folder and double click on Internal. Click Close.
    10. Click Next on the Access Rule Sources page.
    11. On the Access Rule Destinations page, click the Add button.
    12. In the Add Network Entities dialog box, click the Networks folder and then double click on the External network. Click Close.
    13. Click Next on the Access Rule Destinations page.
    14. On the User Sets page, accept the default entry All Users and click Next.
    15. On the Completing the New Access Rule Wizard page, click Finish.

    Configuring the ISA Server 2004 firewall to allow the SMTP relay outbound access to the SMTP protocol

    The SMTP relay computer must be able to accept incoming mail and send outgoing mail. The incoming mail is accepted via a Server Publishing Rule, which we will create later. The outgoing mail is handled by an Access Rule that allows the SMTP relay outbound access to external DNS servers.

    Perform the following steps to create the SMTP Access Rule:

    1. At the ISA Server 2004 firewall computer, open the Microsoft Internet Security and Acceleration Server 2004 management console and expand the server name. Click the Firewall Policy node.
    2. Right click the Firewall Policy node, point to New and click Access Rule.
    3. On the Welcome to the New Access Rule Wizard page, enter a name for the rule in the Access Rule name text box. In this example we’ll call it Outbound SMTP. Click Next.
    4. On the Rule Action page, select Allow and click Next.
    5. On the Protocols page, select the Selected protocols option from the This rule applies to list. Click the Add button.
    6. In the Add Protocols dialog box, click the Common Protocols folder and double click on SMTP. Click Close.
    7. Click Next on the Protocols page.
    8. On the Access Rule Sources page, click the Add button.
    9. In the Add Network Entities dialog box, click the New menu and click Computer.
    10. In the New Computer Rule Element dialog box, enter SMTP Relay in the Name text box. In the Computer IP Address text box, enter 10.0.0.3, which is the IP address of the SMTP relay in this example. Click OK.
    11. In the Add Network Entities dialog box, click the Computers folder and then double click on the SMTP Relay entry. Click Close.
    12. Click Next on the Access Rule Sources page.
    13. On the Access Rule Destinations page, click the Add button.
    14. In the Add Network Entities dialog box, click the Networks folder and then double click on the External network. Click Close.
    15. Click Next on the Access Rule Destinations page.
    16. On the User Sets page, accept the default entry All Users and click Next.
    17. On the Completing the New Access Rule Wizard page, click Finish.

       

    Creating an SMTP Server Publishing Rule Allowing Inbound Connections to the SMTP Relay on the Internal Network

    A Server Publishing Rule allows external hosts to access servers located behind the ISA Server 2004 firewall. A Server Publishing Rule can either route, or perform a reverse NAT to forward the incoming connections to the published server. In the case of the published SMTP server in this example, the publishing rule performs reverse NAT and exposes the connection to the SMTP filter, which protects the SMTP server from buffer overflow attacks.

    Perform the following steps to publish the SMTP server:

    1. In the Microsoft Internet Security and Acceleration Server 2004 management console, expand the server name and then click on the Firewall Policy node.
    2. Right click on the Firewall Policy node, point to New and click Server Publishing Rule.
    3. On the Welcome to the New Server Publishing Rule Wizard page, enter a name for the rule in the Server publishing rule name text box. In this example, we’ll name the rule Publish SMTP Relay. Click Next.
    4. In the Select Server dialog box, enter the IP address of the SMTP relay in the Server IP Address text box. In this example, the SMTP relay is at IP address 10.0.0.3. We will enter that address into the text box and click Next.
    5. On the Select Protocol page, select the SMTP Server protocol from the Selected protocol list. Note that you do not need to publish the secure SMTP server protocol separately, since the SMTP relay is able to accept incoming SSL/TLS secured connections on TCP port 25. Click Next.
    6. On the IP Addresses page, select the External network by placing a checkmark in the External checkbox. Click Next.
    7. Click Finish on the Completing the New Server Publishing Rule Wizard page.
    8. Click Apply to save the changes and update the firewall policy.
    9. Click OK in the Apply New Configuration dialog box.

    Configuring Public DNS entries to support the SMTP Server Publishing Rule

    Remote SMTP clients need to be able to resolve the name on the SMTP server’s certificate to the IP address on the external interface of the ISA Server 2004 firewall machine. The reason for this is that the SMTP client application (such as Outlook or Outlook Express) is configured to use the FQDN contained on the certificate to verify the identity of the server. If the name on the certificate does not match the name the SMTP client application uses to connect to the SMTP server, the connection attempt will fail.

    The is a very common reason for SSL/TLS connection failure. It is critical that you configure your public DNS with the same name on the SMTP virtual server’s certificate. In the current example, we chose the name mail.msfirewall.org. The external SMTP server client must be able to resolve this name to the IP address on the external interface of the ISA Server 2004 firewall machine you’re using to publish the Web site.

    This is quite easy to do if you’re hosting your own DNS servers. If you have a third party host the DNS records for your domain, its may be easy or virtually impossible to get the correct records put into place.

    If you manage your own DNS servers and those DNS servers use the Microsoft Windows Server 2003 DNS server (which believe it or not, is one of the most secure DNS servers available, just check for BIND DNS exploits and compare the number of those with the number of identified Microsoft DNS server exploits and you’ll get the point pretty quick), then you can perform the following procedures to create the required records.

    The first record you must create is a Host (A) record that maps the IP address on the external interface of the ISA Server 2004 firewall to the name on the SMTP virtual server’s certificate. In this example, the name is mail.msfirewall.org, so we need to create a Host (A) record with the same name. Perform the following steps to create the Host (A) record:

    1. On the Windows Server 2003 DNS server, click Start and point to Administrative Tools. Click DNS.
    2. In the DNS console, expand your server name and then expand the Forward Lookup Zones node. Right click on the domain name and click New Host (A).
    3. In the New Host dialog box, enter the host name portion of the FQDN listed on the SMTP virtual server’s certificate in the Name (uses parent domain if left blank text box. In this example, the host name portion of the FQDN is mail, so we will enter that into the text box. In the IP address text box, enter the IP address on the external interface of the ISA Server 2004 firewall that you used in the SMTP Server Publishing Rule. In this example, the external IP address is 192.168.1.70, so we enter that into the text box. Click Add Host.

    1. Click OK in the DNS dialog box informing you that the record was successfully created.
    2. Click Done.

    The next step is to create an MX record for your e-mail domain. Perform the following steps on the Windows Server 2003 DNS server to create the MX record:

    1. Right click the domain name and click the New Mail Exchanger (MX) command.
    2. In the New Resource Record dialog box, enter the FQDN of the Host (A) record you created for the SMTP relay server. In this example, the name is mail.msfirewall.org. Enter this value in the Fully qualified domain name (FQDN) of mail server text box. You can also use the Browse button to find this host record.

     

    1. Click OK in the New Resource Record dialog box.

    Configuring authoritative DNS server entries with your domain Registrar

    If you host your own DNS servers (and even if you don’t), you must inform your domain Registrar of at least two DNS servers that are authoritative for your domain. You can configure your DNS all day long and have the configuration absolutely perfect, but if you do not inform your domain Registrar, then no one will be able to locate your DNS servers to obtain information about the resources you host.

    The procedure for informing your domain Registrar varies among the Registrars. I use Network Solutions, or Verisign, or whatever name they go by today J . I go to their Web site and enter the names of my DNS servers and the IP addresses that go with those names. Go Daddy, another Domain Registrar, has a similar procedure, but it appears to be much simpler. If you publish your own DNS servers, make sure to have at least two IP addresses bound to the external interface of the ISA Server 2004 firewall and publish two Internal DNS servers (you are required to have two public DNS servers for fault tolerance).

    In a later article, we’ll go over the procedures for publishing DNS servers, and I’ll also share with you some tips and tricks on how to make your DNS publishing scheme as fault tolerant as possible. But this article is already getting too long, so we’ll move on to the next step, which is installing the root CA certificate into the user’s Trusted Root Certification Authorities certificate store.

    Install the Root CA Certificate on the SMTP Client Computer

    The SMTP client computer must have the root CA certificate installed into the local user’s certificate store. The reason for this is that there is no interface that allows you to continue to connect to the secure SMTP virtual server when using a e-mail client application like Outlook or Outlook Express.

    If you’ve ever tried to connect to a Web site that uses a secure SSL link, but you do not have a CA certificate installed for the root CA that issued that Web site’s certificate, then you know that one of issues that appears in the information dialog box that pops up is that you have not chosen to trust the CA that issued the site’s certificate. This dialog box allows you to continue, but you do not have this option when using an SMTP client application. Therefore, you must install the CA certificate on the client machine before you attempt to create a secure SMTP connection to the SMTP relay machine.

    Perform the following steps to obtain and install the root CA certificate into the user’s Trusted Root Certification Authorities store:

    1. On the Outlook Express e-mail client computer, enter the IP address and path that connects you to the enterprise CA’s Web enrollment site. You can do this from an external host by publishing the Web enrollment site, you can you connect to the machine directly if the host is located behind the ISA Server 2004 firewall. In this example, we’ll assume the client is located behind the ISA Server 2004 firewall when it requests and installs the enterprise CA’s root certificate. Enter http://10.0.0.2/certsrv in the Address bar and press ENTER.
    2. In the Connect to dialog box, enter Administrator in the User name text box and the Administrator’s password in the Password text box. Click OK.
    3. On the Welcome page of the Microsoft Certificate Services site, click the Download a CA certificate, certificate chain, or CRL link.
    4. On the Download a CA Certificate, Certificate Chain, or CRL page, click the Install this CA certificate chain link.
    5. Click Yes in the Security Warning dialog box asking if you want to install the Microsoft Certificate Enrollment Control.
    6. Click Yes in the Potential Scripting Violation dialog box informing you that the Web site will add a certificate to the machine.
    7. Click Yes in the Root Certificate Store dialog box asking if you want to add the CA certificate.
    8. Close the browser after you see the CA Certificate Installation page that informs you that The CA certificate chain has been successfully installed.

    Configure the SMTP Client and Test the Connection

    Yes! We’re just about ready to test the connection. The SMTP relay and the Exchange Server are ready to go. The ISA Server 2004 firewall is setup with the required Access Rules and an SMTP Server Publishing Rule. The last link in the chain is the SMTP client application.

    Any e-mail client application can be an SMTP client. Outlook, Outlook Express, and all other popular e-mail client applications can be configured as SMTP clients. However, there may be varying support for SSL/TLS secured SMTP connections and the configuration interface varies widely between products.

    For this reason, I’ll demonstrate how to configure the secure SMTP client with Outlook Express. Outlook Express is almost universal, since its included with every version of Windows. Outlook Express is an effective, albeit, not full featured e-mail client, and many corporate and home users find it an acceptable alternative to the full Outlook 2000/2002/2003 client.

    The CA certificate is already installed on the Outlook Express client computer. The next step is to create the e-mail account and then configure the e-mail account to log into the SMTP server using NTLM and also to use SSL/TLS for the connection.

    Perform the following steps to create the e-mail account in Outlook Express:

    1. At the Outlook Express client machine, open Outlook Express.
    2. If the New Account Wizard starts, close it. This Wizard only comes up when no account has ever existed on the machine. I want these instructions to apply to everyone, whether they have created an account on the machine before or not.
    3. Click the Tools menu and then click Accounts.
    4. In the Internet Accounts dialog box, click the Mail tab.
    5. On the Mail tab, click the Add button. From the fly-out menu, click the Mail command.
    6. On the Your Name page, enter your name or alias. In this example, we will use Administrator. Click Next.
    7. On the Internet E-mail Address page, enter your e-mail address in the E-mail address text box. In this example, we will use [email protected]. Click Next.
    8. On the E-mail Server Names dialog box, select the appropriate type of mail server from which the client pulls down new mail. We have not covered POP3 or IMAP4 publishing in this article, so neither of these options will work at this time. However, it is very simple to publish both of these services. Check out the ISA Server 2004/Exchange Deployment Kit for details on how to publishing the Exchange POP3 and IMAP4 services. We will use the name mail.msfirewall.org for the Incoming mail (POP3, IMAP or HTTP) server and we will use the same name for the Outgoing mail (SMTP) server. We will select the IMAP option in the My incoming mail server is aserver list. Click Next.
    9. On the Internet Mail Logon page, enter the user account name and user password in the Account name and Password text boxes. In this example the account name is Administrator and the administrator’s password is entered. Note that this setting only applies to the IMAP service logon, not the SMTP service log on.
    10. On the Congratulations! page, click Finish.
    11. On the Mail tab, click the new mail account and click Properties.
    12. On the mail account’s Properties dialog box, click the Servers tab.
    13. On the Servers tab, put a checkmark in the My server requires authentication checkbox that is located in the Outgoing Mail Server frame. Click the Settings button.

    14. In the Outgoing Mail Server dialog box, select the Log on using option. In the Account name text box, you can enter the user name using the UPN (like [email protected]) or the DOMAIN\username format. We will enter the name [email protected] in the Account name text box and enter the Administrator’s password in the Password text box. Put a checkmark in the Remember password checkbox, and put a checkmark in the Log on using Secure Password Authentication checkbox. Click OK.

    15. Click the Advanced tab.
    16. On the Advanced tab, put a checkmark in the This server requires a secure connection (SSL) checkbox that lies just under the Outgoing mail (SMTP) port number text box.

    1. Click Apply and then click OK in the mail accounts Properties dialog box.
    2. Click Close in the Internet Accounts dialog box.
    3. If you selected the IMAP server as your download server, you will see a dialog box asking if you want to download folders now. Click No.
    4. Close Outlook Express.

    Now we’re ready to do some testing. First, let’s test our ability to send mail to the msfirewall.org domain. Remember that users do not need to authenticate to send mail to this domain because we have configured a remote domain to forward mail to the msfirewall.org domain to the Exchange Server. This test will just test basic functionality of the SMTP Server Publishing Rule:

    1. Create an e-mail message in Outlook Express and address the message to [email protected] or an account in your domain. In the subject line enter Test and in the body enter test. Send the message.
    2. The message arrives at the Exchange Server. If you had real-time monitoring turned on, or if you query the log files at the ISA Server 2004 firewall, you will see that the Server Publishing Rule accepted the connection

    Next, we’ll test the ability of the machine to relay through the SMTP relay to an external domain. Remember that in order to relay through to an external domain, the SMTP client application must be able to authenticate with the SMTP relay machine. This will test the ability to relay through the SMTP server, check the authentication method, and confirm that SSL/TLS is being used to secure the connection. I will run Network Monitor on the SMTP relay machine to confirm these findings:

    1. Create an e-mail message on the Outlook Express computer and address it to an external user. I’ll address this message to [email protected] and enter Test in the subject line and Test in the body. Send the message.
    2. In Network Monitor, you can see the following frames. The first figure shows the SMTP relay machine sending the requirements for establishing the session to the Outlook Express client. The second figure shows the Outlook Express client sending the STARTTLS command to the SMTP relay machine. After this point, the communications are secured by SSL/TLS encryption and you are unable to see the user credentials or the data in the Network Monitor trace.

    1. However, if you look at the Event Viewer’s Security node, you can see the log on of the Outlook Express client, as seen the lines below. NTLM authentication was used and the IP address of the Outlook Express client is listed in the Source Network Address line.

    Event Type: Success Audit

    Event Source: Security

    Event Category: Logon/Logoff

    Event ID: 540

    Date: 6/2/2004

    Time: 11:12:52 AM

    User: MSFIREWALL\Administrator

    Computer: SMTPRELAY

    Description:

    Successful Network Logon:

    User Name: Administrator

    Domain: MSFIREWALL

    Logon ID: (0x0,0xC13F3)

    Logon Type: 3

    Logon Process: NtLmSsp

    Authentication Package: NTLM

    Workstation Name: XPPROSP1

    Logon GUID: –

    Caller User Name: –

    Caller Domain: –

    Caller Logon ID: –

    Caller Process ID: –

    Transited Services: –

    Source Network Address: 192.168.1.71

    Source Port: 1436

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    You can also view the e-mail headers of this message to see the path taken to the external domain. Notice that the header on the SMTP relay machine indicates that a secure channel was used to send the initial message.

    Microsoft Mail Internet Headers Version 2.0

    Received: from spamwhacker3.shindermail.net ([192.168.1.102]) by owa.shindermail.net with Microsoft SMTPSVC(6.0.3790.0);

    Wed, 2 Jun 2004 11:53:52 -0500

    Received: from MAILSEC8 ([192.168.1.101]) by IHATESPAM.shindermail.net with Microsoft SMTPSVC(5.0.2195.6713);

    Wed, 2 Jun 2004 11:53:49 -0500

    Received: from ME8 ([192.168.1.100]) by MAILSEC8 with Microsoft SMTPSVC(5.0.2195.5329);

    Wed, 2 Jun 2004 11:55:31 -0500

    Content-Class: urn:content-classes:message

    Received: from FILEWAY ([192.168.1.21]) by ME8 with Microsoft SMTPSVC(5.0.2195.5329); Wed, 2 Jun 2004 11:53:02 -0500

    Received: from SMTPRELAY.msfirewall.org ([209.30.88.108]) by FILEWAY with Microsoft SMTPSVC(5.0.2195.2966); Wed, 2 Jun 2004 11:53:12 -0500

    Received: from xpprosp1 ([192.168.1.71]) by SMTPRELAY.msfirewall.org over TLS secured channel with Microsoft SMTPSVC(6.0.3790.0); Wed, 2 Jun 2004 11:52:42 -0500

    You can also see the outgoing message by viewing the ISA Server 2004 firewall’s real-time log monitor. The Publishing SMTP Relay lines indicate the incoming connections to the SMTP relay machine via the SMTP Server Publishing Rule and the Outbound SMTP lines indicate the outbound SMTP messages being sent to the external domain.

    The last test we’ll perform is to test the MailEssentials spam whacker. MailEssentials only examines incoming mail destined to one of the remote domains (called “local domains” by the MailEssentials software) configured on the SMTP relay. Mail that authenticated users send to external domains is not screened by MailEssentials. However, if you have such a need, the SMTP message screener will block mail going in both directions, although the SMTP Message Screener lacks the flexibility and power of MailEssentials. Note that MailEssentials does examine the outgoing mail to help create its “HAM” list of legitimate e-mail massages used by its Bayesian filter.

    The following test shows what happens if we use the default keyword entries in MailEssentials.

    1. At the Outlook Express client, create an e-mail message addressed to [email protected] and include in the subject line the word casino and in the body the word casino.
    2. Because the word casino is included in the keyword list on MailEssentials, the mail is held in the folder we specified.

    Note that MailEssentials allows you a great number of options for blocking spam, and you should check into these various options. In this article I wanted to demonstrate how easy it is to install MailEssentials on a secure SMTP relay machine that can accept anonymous mail from Internet SMTP servers and secure, authenticated connections from your remote users.

    Get the New Book!

    Summary

    In this article we discussed the detailed procedures you can carry out to publish an SMTP relay machine that can accept secure, authenticated connections from your remote users, and anonymous non-secured connections from Internet e-mail servers. The secure, authenticated connections allow your remote users to relay mail to external domains and send mail to the domains you host. The anonymous, non-secured connections from external SMTP servers allows Internet SMTP servers to send mail to domains you host, and no others.

    I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=23;t=000068 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom

    If you would like us to email you when Tom Shinder releases another article on ISAserver.org, subscribe to our ‘Real-Time Article Update’ by clicking here. Please note that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy

    Leave a Comment

    Your email address will not be published.

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

    Scroll to Top