Configuring an ISP Co-located Web/SMTP/ISA Server

Configuring an ISP Co-located Web/SMTP/ISA Server

By Thomas W Shinder M.D.

One question that comes up every week or so over the last couple of years is whether you could install ISA Server as a unihomed device running multiple server applications and leverage the firewall features of the ISA Server software. Many of the people asking about this type of configuration wanted to install a Windows 2000 Server at an ISP with a co-lo setup. My response usually was that I didn’t think it would make that much difference, since you can use IPSec Policies, RRAS packet filters and TCP/IP Security to lockdown a public server.

However I recently rethought my position, as I had a call from someone who wanted to hire me to set up this type of configuration for him. His reasoning was that ISA Server is a firewall, so it should be able to provide a higher level of protection, and he also wanted to leverage the features provided by Web and Server Publishing rules. Rather than telling him I didn’t think it would be a good idea, I decided to test it in the lab, and if it worked, we would go ahead and set him up. It worked in the lab and its working for him now in a live ISP co-lo setup.

The procedure is fairly simple if you’re already “ISA Server aware”. Even if you’re not an ISA Server pro, you can still get it working. You need to do the following things:

Let’s look at each of these in more detail.

The first step is always to install and configure the Windows 2000 Server machine. Make sure you install only that services you require. If you don’t need the SNMP service, don’t install it! If you don’t need the DNS service, don’t install it! For some good advice on how to improve security on your ISA Server, check out my ISA Server Security Checklist, part 1  and ISA Server Security Checklist, part 2.

The basic steps required for our unihomed ISP co-lo Windows 2000 server include:

First you install the Windows 2000 server with the minimal number of services. Then you install the Microsoft Loopback adapter, assign the interfaces the appropriate address and put the primary adapter on top of the Interface list.

Although I say that you’re installing a unihomed Windows 2000 Server, the machine will end up dual-homed. The difference between this scenario and the typical ISA Server setup is that there is only a single physical interface on the server. The physical interface will act as the external interface. The second interface is a virtual interface, which is implemented as the Microsoft Loopback Adapter. The loopback adapter will act as the private interface and its IP addresses will be included in the LAT. Note that you don’t have to use the loopback adapter. You can put in a second physical interface and plug it into a hub and have the same effect.

The Microsoft Loopback adapter can be added via the Add/Remove Programs Applet on the ISA Server computer. Tell the Wizard that you don’t want it to automatically detect the new hardware and to let you pick from a list. Then choose to install a new Network Adapter. Choose Microsoft as the vendor and you’ll see the Loopback adapter as Microsoft only NIC. Install that and then assign it a private IP address.

Remember to perform all these functions while the machine is not connected to the Internet. You’re not protected until you secure the server, and you won’t be secure until you install the ISA Server. Make sure the IP addressing information on the external interface matches what you’ve been assigned by your ISP. You can use any private IP address on the internal interface. You can bind multiple IP addresses to the loopback adapter if you want to publish multiple sites on different IP addresses.

Finally, make sure that you put the loopback adapter interface on the top of the interface list, as seen in the figure below. Note that I’ve renamed the adapters, with the WAN interface being the external interface and the BOGUS interface being the loopback adapter interface.

Socket pooling is the bane of any ISA Server administrator who tries to run services on the ISA Server itself. Just some of the services that require that you disabled socket pooling for include:

The IIS FTP and WWW services use one method to disable socket pooling, and the SMTP/NNTP/POP3/IMAP4 services use a second method. You can find out everything you want to know about disabling socket pooling in my article The Misery of IIS 5.0 Socket Pooling 

You should disable socket pooling before you install ISA Server. If you are planning to install Exchange 2000 on the ISA Server as part of your unihomed ISP co-lo setup, then you should disable socket pooling and configure the sites after you install Exchange, but before you install ISA Server. We won’t go over all the details or subtleties of an Exchange 2000 setup on the ISA Server itself in this article. You can get all the juicy details and step by steps in the ISA Server and Beyond book.

The next step is to configure the services to listen only on the internal interface. You can configure all the IIS and Mail services to listen on a specific IP addresses. You need to do this manually, because the default setting is to allow the services to listen on all addresses. While that configuration works reasonably well when the IIS or mail server is on the internal network, but doesn’t work when we’re running them on the same machine as the ISA Server and we want to publish those servers to the Internet.

For example, look at the figure below. The (All Unassigned) option is the one selected by default. I will change this to the internal IP address 172.16.0.1. Note that the internal interface has a second IP address assigned to it (in this case, it was actually assigned to the VPN interface of the server). The external interface in this example is 192.168.10.2. You do not use the external IP address when binding the sites to an address. You can add multiple IP addresses to an internal interface if you want to publish multiple sites.

Make the same changes for each service or server you want to support. Check and recheck that you have configured all IIS and mail sites and services to listen on the internal interface only.

After you get the Windows 2000 Server and the Internet services set up, you’re ready to install the ISA Server. There are no special configuration issues in this scenario, except that you probably don’t care out the H.323 Gatekeeper, so you can leave that out of the installation if you wish. Things to take care of at point include:

You must install ISA Server Service Pack 1 as soon as you finish install ISA Server. You should also consider installing the ISA Server Feature Pack 1. This is especially true if you want to use the SMTP filter and Message Screener. The new SMTP filter provides for better performance and supports the AUTH command for incoming SMTP connections. AUTH support is very helpful if you wish to create a virtual SMTP server to support your external clients. Don’t use the OWA Wizard in this scenario. Use the methods described in ISA Server and Beyond.

After installing Service Pack 1, double check your LAT. The only addresses you want in the LAT are those you want to use on the internal interface. Its fine to use the entire private network ID, you don’t need to enter individual addresses. One thing you might want to do is remove the checkmark from the Add the following private ranges… option. Also remember to include the addresses you wish to use for the static address pool, if they differ from the Network ID used by the internal interface.

You’ll need to configure one or more Incoming Web Requests listeners if you plan on doing any kind of Web Publishing. You can configure the Incoming Web Requests listener to use the same configuration for all addresses, or you can configure listeners individually. I highly recommend that you configure each listener individually. This is especially true if you plan to host multiple secure (SSL) servers on the machine. Each certificate will require its own listener.

If you plan on using Web Publishing Rules, then you need to create Destination Sets before creating the Web Publishing Rules. Destination Sets can use either IP addresses or FQDNs, but don’t mix IP addresses and FQDNs in the same Destination Set. However, when it comes to Web Publishing rules, I highly recommend that you do not use IP addresses. While IP addresses do work in Destination Sets used for Web Publishing Rules, they can cause problems that you’re better off avoiding. Microsoft recommends that you use FQDNs and I do too.

I prefer to create Web Publishing Rules that use the FQDN of the site for the redirect. For example, you can configure a Web Publishing Rule to redirect a request to an IP address on the internal network, a NetBIOS name on the internal network, or a FQDN on the internal network. Using FQDNs for the redirect provides for cleaner log files and simplifies many issues related to SSL publishing. For example, we’ll publish a Web site that external users will access by connecting to www.internal.net. In order to support this Web Publishing Rule, we need to create a HOSTS file entry for www.internal.net and map that to the IP address on the internal interface of the ISA Server, as seen in the figure below.

You don’t need to configure protocol rules to support outbound access for your services. For example, if you allow external network users to connect to your SMTP server send mail, the SMTP server component will need outbound access to TCP port 25 to forward the mail to Internet SMTP servers (these users should authenticate to access the SMTP server as you do not want to run an open SMTP relay). Any service installed on the ISA Server that needs to create a new outbound connection requires a packet filter configured to support that connection. You do not need to create packet filters to support Server or Web Publishing Rules; the ISA Server will create a dynamic packet filter to allow it to respond to external clients.

We’re ready to create the Web and Server Publishing Rules now that the basic infrastructure is in place. You can create Web Publishing Rules to publish one or one hundred Web sites. You can also publish virtually any protocol, such as SMTP, NNTP. POP3, IMAP4 and RDP. Lets look at two examples, a Web Publishing Rule and a Server Publishing Rule.

First, let’s publish a Web site accessible via www.internal.net:

1. In the ISA Management console, expand the Publishing node and right click on the Web Publishing Rules node. Point to New and click Rule.
2. In the Welcome to the New Web Publishing Rule Wizard page, type in a name for the rule and click Next.
3. On the Destination Sets page, click the down arrow for the Apply this rule to drop down list box and select the Specified destination set option. In the Name drop down list box, select the name of the Destination Set you created for your Web Publishing Rule. In this case, I created a Destination Set for www.internal.net Click Next.

4. On the Client Type page, select the client type you want to support. In this example I’ll select the Any request option. Click Next.
5. On the Rule Action page, select the Redirect the request to this internal Web server (name or IP address) option and then type in the FQDN you entered for this site in the HOSTS file. In this example, I’ll enter www.internal.net. The Web Proxy service will resolve the redirection to the internal IP address of the ISA Server to the internal address of the ISA Server that this site is bound to. Click Next.

6. Review your settings and click Finish.

Now let’s create an RDP Server Publishing Rule:

1. The first thing you need to do is configure the Terminal Server to listen on the internal interface. You can do this in the Terminal Services Configuration utility. Click on the Connections node in the left pane of the console and double click on the RDP-Tcp entry in the right pane. Click on the Network Adapter tab and select the Microsoft Loopback Adapter.

2. The next thing you need to do is create an RDP Server Protocol Definition. The primary connection should be configured as TCP port 3389 Inbound. There are no secondary connections.

3. Now to the Server Publishing Rule. Right click the Server Publishing Rules node, point to New and click New.
4. On the Welcome to the New Server Publishing Rule Wizard page, type the name of the rule and click Next.
5. On the Address Mapping page, type in an internal IP address on the ISA Server in the IP address of internal server text box. The Terminal Server will listen on all internal IP addresses on the internal interface, so it doesn’t matter which one you select. In the External IP address on ISA Server click the Browse button and select the IP address you want the RDP server publishing rule to listen on. Click Next.

6. On the Protocol Settings page, select the RDP Server Protocol Definition and click Next.
7. On the Client Type page, select the client type you want to connect to the Terminal Services. Its usually a good idea to create a client address set that has a selected number of IP addresses that you want to allow to access the Terminal Server. Click Next.
8. Review your settings and click OK.

Create all the Web Publishing and Server Publishing Rules you need and then move on to the next step.

Now we want to do the following:

Go to any browser on the external network and test your Web Publishing Rule. Testing from an internal network client isn’t something you can do in this example, since the internal interface is a bogus interface . In the figure below you can see that we’ve establish a connection to the Web site. I’ve also opened a command window below that so you can see that we’ve established a connection to the external interface of the ISA Server.

We’re able to connect to the Terminal Server, as seen in the figure below:

Now what do the Log files show? Let’s take a look at the Web Proxy log:

Not bad. We’re seeing the IP address of the requesting host in the Web Proxy log and it look about the same as you would see in any other Web Publishing situation.

Let’s look at the Firewall service log:

Hmm. This looks a bit more problematic. The source IP address is the internal interface of the ISA Server and not the IP address of the machine that made the request. This is what you’ll always see when you use Server Publishing Rules to publish services running on the ISA Server itself. This makes forensic analysis problematic if you hope to use the Firewall service log to track down connection attempts. You’ll see the same thing in the in files of the service, such as the SMTP, NNTP, POP3 and IMAP4 logs.

You can consider me a convert. In the past, I considered the ISP co-lo configuration a bit of a kludge. But we’ve had no problems putting this solution together in a live environment. The machine hasn’t been compromised, and the performance is excellent. The one thing I was worried about was Web site security, because we weren’t able to take advantage of the URLScan utility. However, we installed the ISA Server feature pack 1 yesterday and there haven’t been any issues with installing and configuring the URLScan utility.

If you’re experienced with ISA Server, you should have no problems replicating the configuration I described in this article. If you interested in more detailed, step-by-step analysis and configuration examples of installing and configuring Exchange 2000 on the ISA Server itself, check out chapter 6 of the ISA Server and Beyond book.