Troubleshooting SMTP Server Publishing Rules

telnet 192.168.1.70 25

and press ENTER. Replace 192.168.1.70 with the IP address you’re using for the listener on your SMTP Server Publishing Rule. If you see what appears in the figure below, you know that it’s not primarily a DNS problem that’s causing the connectivity issues. That isn’t to say that there can’t also be a DNS problem, but at this point we know for sure that there is a problem that isn’t DNS related.

Now we have to wonder if the Telnet connection even made it to the external interface of the ISA firewall. We can quickly check this out by using the ISA firewall’s real-time log viewer. Perform the following steps to configure the real-time log viewer to focus on the connections of interest:

1. In the Microsoft Internet Security and Acceleration Server 2004 management console, expand the server name and click the Monitoring node.
2. On the Monitoring node, click the Logging tab. While on the logging tab, click the Tasks tab in the Task Pane. Under the Logging Tasks list, click the Edit Filter link.
3. In the Edit Filter dialog box, click the down-arrow in the Filter by drop-down list. Click the Protocol entry. Click the down-arrow for the Condition drop-down list and click the Equals entry. Click the down-arrow in the Value drop-down list and select SMTP Server. Click the Add to List button.

4. Click the Start Query button.
5. Telnet to the ISA firewall again. In the ISA firewall’s real-time log viewer, you’ll see something like the figure below. This proves that your telnet connection actually made it to the external interface of the ISA firewall but the connection failed. The Rule column shows that a Server Publishing Rule named SMTP Server allowed the inbound connection to the SMTP server. Notice the Client IP address is 192.168.1.90, which is the IP address of the host on the external network, and the Destination IP is the address of the SMTP server on the Internal Network.

At the point we know the connections are making it to the ISA firewall from our test client on the external network and we know that the Server Publishing Rule is correctly configured. The next step is to use Network Monitor to see what might be happening to these connections.

We can use Network Monitor on the ISA firewall and on the Exchange Server to arrive at a solution for this problem. You’ll find that using Network Monitor can solve the vast majority of ISA firewall and related problems you encounter.

The first step is to install Network Monitor, which is not installed by default. You can install Network Monitor using the Add/Remove Programs applet in the Control Panel. If you haven’t already done so, install Network Monitor on both the ISA firewall and the Exchange Server.

Perform the following steps on the ISA firewall after installing Network Monitor:

1. On the ISA firewall machine, click Start and point to All Programs. Point to the Administrative Tools menu and click Network Monitor.
2. The first thing you’ll see is a request for you to specify a network for Network Monitor to listen on. Click OK in the Microsoft Network Monitor dialog box.

3. In the Select a network dialog box, expand the Local Computer node in the left pane. Under the Local Computer node you’ll see a list of network interfaces on the ISA firewall machine. Notice that the interfaces in this example have meaningful names. The names are those I gave the interfaces by renaming the interfaces in the Network and Dial-up Connections window. If you didn’t rename the interfaces, then you’ll see something like Local Area Connection 1, Local Area Connection 2, etc. You’ll then need to figure out which of the Local Area Connections represent which interface. In this example we want Network Monitor to listen on the LAN interface. Click the interface representing the network interface closest to your Exchange Server and click OK.

4. Click the Capture menu and click the Buffer Settings command.
5. In the Capture Buffer Settings dialog box, enter 100 in the Buffer Size (MB) text box. This will help prevent missing packets we’re interested in because we ran out of buffer space for the capture. Click OK.

6. Start the capture by clicking the Start Capture button, which looks like a right-pointing arrow in the button bar.
7. Go back to the test client on the external network and run the telnet command again.
8. Return to the ISA firewall and click the Stop and View Capture button in the console’s button bar. This button looks like a square with a pair of eye-glasses above the square.
9. If you have a busy firewall, your capture file can get quite large. You can create a display filter to help simplify things. In the [Capture 1: (Summary)] window, click the Display menu and then click Filter.
10. In the Display Filter dialog box, click the Protocol == Any entry and click Edit Expression.

11. In the Expression dialog box, click the Property tab. On the Property tab, scroll down to find the TCP entry in Protocol Property list. Double click TCP and click the Destination Port entry. Click the = entry in the Relation list. Select the Decimal option and then enter 25 in the Value (Word) text box. Click OK.

12. Click OK in the Display Filter dialog box.

13. In the figure below you can see the filtered list of SMTP connections. The figure shows that our external host at 192.168.1.90 is attempting a connection to IP address 10.0.0.2. This is similar to the information we gathered from the real-time log viewer. However, what is significant here is that these packets are leaving the internal interface. In contrast, what we saw in the real-time log viewer was the connection attempt on the external interface. Since the Network Monitor is configured to listen on traffic on the internal interface, we only see traffic handled by that interface. In this case, we see traffic leaving the internal interface and destined to the Exchange Server’s IP address. This indicates the ISA firewall’s Server Publishing Rule is doing what it’s supposed to do.

Since the ISA firewall’s Server Publishing Rule is working correctly, the next step is to run Network Monitor on the Exchange Server. Install Network Monitor on the Exchange Server and then perform the following steps:

1. At the Exchange Server, click Start and point to All Programs. Point to Administrative Tools and click Network Monitor.
2. Click OK in the dialog box asking you to configure a network on which you want to capture data.
3. In the Select a network dialog box, expand the Local Computer node in the lane pane and click the Local Area Connection entry (since the Exchange Server would only have one NIC, there’s no need to rename the interfaces). Click OK.
4. Click the Capture menu and then click the Buffer Settings command.
5. In the Capture Buffer Settings dialog box, enter 100 in the Buffer Size (MB) text box. Click OK.
6. Click the Start Capture button in the Network Monitor button bar.
7. Run the telnet command on the external client.
8. Return to the Exchange Server and click the Stop and View Capture button.
9. Since we’re not really sure what we’re looking for here, we won’t filter this capture. You’ll see something like what appears in the figure below. The capture shows that the incoming connections are arriving to the Exchange Server. After each connection we see an ARP broadcast seeking the MAC address of 10.0.0.10. These ARP Request messages don’t have any match ARP Reply messages. This indicates that host 10.0.0.10 is either offline or doesn’t exist. The question you need to ask yourself at this point is “why is the Exchange Server trying to find this IP address”?

The reason why there are no ARP replies in this example is that there is no host with the IP address 10.0.0.10. So why is the Exchange Server looking for 10.0.0.10 each time the external host connects to the Exchange Server? The Exchange Server is trying to reply to the external client by connecting to its default gateway. The response must go to the Exchange Server’s default gateway because the destination IP address of the test client is on a different network ID than the Exchange Server. In this example the Exchange Server was configured with 10.0.0.10 as its default gateway, probably because of a typo when configuring the default gateway setting on the Exchange Server’s network interface.

There are two ways we can solve this problem:

The default configuration for Server Publishing Rules requires the published server to be a SecureNAT client. This is similar to what we saw with ISA Server 2000. However, you have a choice with the new ISA firewall: you can preserve the source IP address of the remote client (the default setting), or you can replace the remote client’s IP address with the IP address of the ISA firewall’s internal interface. When you choose the latter option, the published server does not need to be a SecureNAT client.

Let’s see what happens when we change the default gateway on the Exchange Server to be the IP address on the internal interface of the ISA firewall. After making the change, we can see that the incoming connections make it to the Exchange Server and the outgoing responses are returned to the test host on the external network. The responses made their way outbound because the new default gateway configured on the Exchange Server enabled it to return responses to the host on a different network ID.

At the test client on the external network, you’ll see something like what appears below in the Telnet window.

Another option is to configure the Server Publishing Rule to replace the client IP address. To do this, double click the SMTP Server Publishing Rule and click the To tab. On the To tab, select the Requests appear to come form the ISA Server computer option. Click Apply and then click OK. Then click OK to save the changes to the firewall policy. Click OK in the Apply New Configuration dialog box.

Repeat the telnet connection again while running Network Monitor on the Exchange Server. Now you see the client address sending the SMTP message to the Exchange Server as 10.0.0.1, which is the IP address on the internal interface of the ISA firewall. The Exchange Server responds by sending the replies to 10.0.0.1, since that is the client IP address seen by the Exchange Server. In this case, there is no need to make the Exchange Server a SecureNAT client, because the Exchange Server is not responding to a host on an external network.

The advantage of replacing the client IP address with the address of the internal interface of the ISA firewall is that you don’t need to make the Exchange Server a SecureNAT client, and you don’t have to make any changes in your routing infrastructure to support SecureNAT clients that are on networks remote from the ISA firewall. The downside of this configuration is that you won’t have the actual remote client’s IP address in the SMTP log, although it does appear in the ISA firewall’s Firewall Service log.

In this article we discussed how you can use the Telnet, the ISA firewall’s real-time log viewer, and Network Monitor to troubleshoot an SMTP Server Publishing Rule. These same methods and principles can be used to troubleshoot any other type of Server Publishing Rule. In the example discussed in this article, the Exchange Server was not configured as a SecureNAT client, so its replies could not be returned to the external client. We went over two solutions to this problem: one was to make the Exchange Server a SecureNAT client and the second was to configure the Server Publishing Rule to change the client IP address to the address used on the internal interface of the ISA firewall.

I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=22;t=000168 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom

If you would like us to email you when Tom Shinder releases another article on ISAserver.org, subscribe to our ‘Real-Time Article Update’ by clicking here. Please note that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy.