Configuring Fault Tolerance and Load Balancing for Windows 2003 ISA Firewall/VPN Servers

Configuring Fault Tolerance and Load Balancing for Windows 2003 ISA Firewall/VPN Servers
By Thomas W Shinder, M.D.

You can configure Windows Server 2003 based ISA firewall/VPN servers for high availability by taking advantage of the Windows Server 2003 Network Load Balancing (NLB) service. The NLB service provides two major features that aid in increasing the availability of VPN connections for your VPN clients:

Fail over when one of the ISA firewall/VPN servers
Fail over allows other members of an ISA firewall/VPN server array to service connection requests from VPN clients when one of the servers becomes unavailable. All VPN servers in the array “listen” for VPN connections on the same IP address. When a VPN session is disconnected after a VPN array members goes offline, the connection is reestablished to another array member using the same IP address. The VPN user does not need to reconfigure the VPN client software to automatically reestablish the connection.
Load balancing for VPN connections
VPN sessions can be processor intensive. Data encryption and decryption can take a significant percentage of the processor cycles available to the ISA firewall/VPN server per unit time. The NLB service can automatically split connections across all array members so that no single member of the array receives a disproportion number of connection requests. NLB attempts to even spread the connection requests across all members of the NLB ISA firewall/VPN server array.

Note: A detailed description of the NLB protocol and how it works is beyond the scope of this article. For more information on how NLB works and how to customize the NLB configuration for non-VPN purposes, please refer to the Windows Server 2003 Help file.

In this article we’ll discuss the following:

A description of the example VPN network used in this article
Configuring an NLB array
Installing ISA Server 2000 on the Windows Server 2003 NLB array members
Running the ISA Server 2000 VPN Wizard on the NLB array members
Configuring the ISA Server 2000 packet filters to support connections to the array address

The Example VPN Array Network

Figure 1 shows the details of our example VPN Array Network. Note that I’ve only listed the relevant players in the configuration. There may be dozens, hundreds or even thousands of machines on the network. The network diagram only includes the machines that are important to get the configuration working. You might want to print the network diagram on a separate page so that you can refer to it while you read this article or when testing the procedures in your lab.

Note: It’s important to have the example network easily available while you read this document. I do not expect you to memorize the diagram prior to carrying out the procedures!

Figure 1

The ISA firewall/VPN servers have only Windows Server 2003 and ISA Server 2000 installed. No extraneous Windows services and no third party applications are installed on the NLB array members. All machines are members of the same Windows Server 2003 Active Directory domain.

The domain controller on the internal network has the following services installed:

WINS

WINS is not a required networking service. However, if you wish to allow the VPN clients to browse for servers on the internal network, a WINS server will simplify the process.

A DNS server is required on an Active Directory network. VPN clients can be assigned a DNS server address via DHCP when the DHCP Relay Agent is installed and configured to support your VPN clients. See Using DHCP with ISA/VPN Server Clients

DHCP

The DHCP server assigns addresses to internal network hosts and to VPN clients. You can configure the DHCP server to assign custom DHCP options (such as WINS and DNS server addresses and primary domain name) by using a DHCP Relay Agent on the ISA firewall/VPN server.

RADIUS

The RADIUS server can centralize RRAS policy across all the VPN array members. The RADIUS server simplifies the task of creating RRAS policy so that you can create a single policy on the RADIUS server and have that policy apply to all the VPN array members. RADIUS also allows you to use Active Directory domain user accounts without requiring the VPN array members to be part of the same Active Directory domain. Please see the ISA Server 2000 VPN Deployment Kit articles on configuring RADIUS to support VPN clients (the VPN Deployment Kit beta will be available on this site next week).

Active Directory

Active Directory is required on Windows Server 2003 domain controllers.

Configuring the NLB Array

Windows Server 2003 Standard, Enterprise and Datacenter editions support the Windows Server 2003 Network Load Balancing service. One of the major improvements to the NLB service included with Windows Server 2003 is the new Network Load Balancing Manager. The NLB Manager allows you to create, configure and manage NLB arrays using an intuitive graphical interface.

Create the array after you have installed the Windows Server 2003 software on the machines who will be members of the ISA firewall/VPN server array, but before you enable the Routing and Remote Access service with the ISA Server 2000 VPN wizard.

Perform all array management tasks from LOCALISAVPN1. Perform the following steps to create the Windows Server 2003 NLB arrays:

Click Start, point to Administrative Tools, and click on Network Load Balancing Manager (figure 2).

Figure 2

The Network Load Balancing Manager console opens (figure 3). There are no NLB arrays configured by default. You will need to create an NLB array that allows all of the ISA firewall/VPN servers to listen on a single IP address on the external interface.

Figure 3

Click the Cluster menu and click the New command (figure 4).

Figure 4

Fill in the following information in the Cluster Parameters dialog box (figure 5):

IP address

This is the virtual IP address used by all of the members of the NLB array. The NLB Manager will automatically bind this address to the external interface of all the array members

Subnet mask

This is the subnet mask for the virtual IP address

Full Internet name

This is the Fully Qualified Domain Name used to access the cluster IP address for command line remote administration. Enter a name here if you choose to allow command line remote administration. This name must also be entered into the public DNS

Cluster operation mode

The Windows Server 2003 NLB service can operate in either Unicast or Mulicast mode. Choose multicast mode unless you have Cisco routers or switches on the same network segment as the external interface and those routers or switches do not support mapping unicast IP addresses to multicast MAC addresses. Please refer to the Windows Server 2003 Help for more information about NLB, unicast and multicast modes.

Allow remote control

Put a checkmark in this checkbox if you wish to allow command line remote control of the NLB array parameters. We do not wish to allow command line remote control on the external interface array. Do not enable this checkbox.

Remote password

If remote command line administration were available, you would enter a password in this text box.

Confirm password

If remote command line administration were available, you would confirm the password in this text box.

Click Next.

Figure 5

You can add more virtual IP addresses to the array in the Cluster IP Addresses dialog box (figure 6). Click the Add button to add more VIPs. In this example we will not use additional VIPs. Click Next.

Figure 6

A default rule appears in the Port Rules dialog box (figure 7). You can create customized port rules that determine how connections are load balanced across all the servers in the array. Click on the default port rule, and then click the Edit button.

Figure 7

The details of the default port rule appear in the Add/Edit Port Rule dialog box (figure 8). The default port rule includes the following parameters:

Cluster IP address

This entry determines what IP address this rule applies to. The default port rule applies to all addresses in the NLB array

Port range

This entry determines what inbound ports the rule applies to. The default port rule applies to all inbound ports

Protocols

You can have the rule apply to TCP, UDP or Both. The default port rule applies to both TCP and UDP protocols. Note that the Windows Server 2003 NLB port rules can only be applied to TCP and UDP protocols. You cannot apply port rules to other protocols such as ICMP.

Filtering mode

There are three filtering modes:

Multiple host

Specifies whether multiple hosts in the cluster handle network traffic for the associated port rule. The default port rule applies to all hosts in the array and the Affinity setting is set to Single.

Single host

Specifies that network traffic for the associated port rule be handled by a single host in the cluster according to the specified handling priority. This filtering mode provides port specific fault tolerance for the handling of network traffic.

Disable port range

Specifies whether all network traffic for the associated port rule will be blocked.

Please refer to my articles on the Windows 2000 NLB here at www.isaserver.org for more details on NLB. These include:

Using NLB with ISA Server, Part 1: How Network Load Balancing Works http://www.isaserver.org/tutorials/basicnlbpart1.html
Using NLB with ISA Server Part 2: Layer 2 Fun with Unicast and Multicast Modes http://www.isaserver.org/articles/basicnlbpart2.html
Using NLB with ISA Server Part 3: Configuring NLB Array Parameters http://www.isaserver.org/tutorials/nlbpart3.html

Figure 8

Click Next on the Port Rules page (figure 9)

Figure 9

Type in the name of the machine you are running the NLB Manager application on in the Host text box on the Connect page. In this example, we are running the NLB Manager on LOCALISAVPN1. Click the Connect button (figure 10). You will see a list of interfaces on this machine in the Interface available for configuring a new cluster list. Click on the external interface of the ISA firewall/VPN array member. In this example, the external interface is named WAN (this is the name that appears in the Network and Dial-up Connections window; we have renamed the interfaces to make them more descriptive). Click Next.

Figure 10

The details of the NLB array member appear on the Host Parameters page (figure 11).

Priority

Specifies a unique ID for each host.

IP address

This is the IP address on the external interface of the NLB array member for traffic not associated with the cluster (for example, Telnet access to a specific host within the cluster). Type the IP address in standard Internet dotted notation (for example, w.x.y.z). This IP address is used to individually address each host in the cluster and hence should be unique for each host.

Subnet mask

This is for the subnet mask for the IP address specified. Type the mask in standard Internet dotted notation (for example, 255.255.255.0).

Default state

Specifies the default host state of the Network Load Balancing cluster when Windows is started. Select Started option if you want the host to immediately join the cluster when Windows is started. Select the Stopped option if you want this host to start without joining the cluster. Select the Suspended option if you want this host to start without joining the cluster and instead enter a suspended state.

Retain suspended state after computer restarts

Specifies whether the host will remain suspended when Windows is restarted when the host was suspended prior to shutting down.

Click Finish.

Figure 11

You can see the details of the NLB array configuration in the log entry pane in the bottom of the console window (figure 12).

Figure 12

The next step is to add a second machine to the array. Right click the name of the array in the left pane of the Network Load Balancing Manager console and click the Add Host to Cluster command (figure 13).

Figure 13

On the Connect page, type in the name of the computer you want to add to the array in the Host text box. In this example we want to add LOCALISAVPN2 to NLB array (figure 14). Select the external interface of this second array member in the Interface available for configuring the cluster list. Click Next.

Figure 14

The Host Parameters page has the following settings (figure 15):

Priority

Specifies a unique ID for each host.

IP address

Subnet mask

This is for the subnet mask for the IP address specified. Type the mask in standard Internet dotted notation (for example, 255.255.255.0).

Default state

Retain suspended state after computer restarts

Specifies whether the host will remain suspended when Windows is restarted when the host was suspended prior to shutting down.

Click Finish.

Figure 15

You can see the details of the array configuration in the log entry pane at the bottom of the console (figure 16). Double click on the log entry with the description Update 2 succeeded [double click for details…].

Figure 16

The log entry provides verbose details associated with that entry (figure 17). Click OK and close the Network Load Balancing Manager console.

Figure 17

Installing ISA Server 2000 on the Windows Server 2003 NLB Array Members

ISA Server 2000 must be installed on each member of the ISA firewall/VPN array. There are array specific configuration requirements. Please refer to my article on how to install ISA Server 2000 on a Windows Server 2003 machine

Running the ISA Server VPN Wizard on the Windows Server 2003 NLB Array Members

ISA Server 2000 includes a VPN server Wizard that enables the Routing and Remote Access Service and configures ISA Server packet filters that allow access to both PPTP and L2TP/IPSec VPN clients. The ISA Server 2000 VPN server wizard performs most of the required tasks. However, you should customize the settings made by the VPN wizard to meet the requirements of your own network.

Please see Configuring ISA Server For Inbound VPN Calls for instructions on how to run the ISA Server 2000 VPN Wizard.

Configuring the ISA Server 2000 Packet Filters to Support the NLB Array Address

The ISA Server 2000 VPN Wizard automatically configures packet filters that allow PPTP and L2TP/IPSec VPN clients to connected to your ISA firewall/VPN server. However, these packet filters allow inbound VPN client access to the primary IP address bound to the external interfaces on the ISA firewall/VPN server array members. The VIP (virtual IP address) used by the Windows Server 2003 NLB service is not configured as the primary IP address and these default VPN packet filters will fail.

You will need to change these packet filters so that they support connections to the NLB VIP IP address. Perform the following steps on each member of the ISA firewall/VPN array:

Open the ISA Management console. Expand the Servers and Arrays node, then expand your server name. Expand the Access Policy node and click on the IP Packet Filters node (figure 18). Notice in the right pane of the console that the ISA Server 2000 VPN server Wizard has created four VPN related packet filters. Double click on the Allow PPTP protocol packets (server) packet filter.

Figure 18

Click on the Local Computer tab in the Allow PPTP protocol packets (server) Properties dialog box (figure 19). Select the This ISA server’s external IP address option and type in the IP address of the VIP in the text box. Click Apply and then click OK.

Figure 19

Click on the Local Computer tab in the Allow PPTP protocol packets (client) Properties dialog box (figure 20). Select the This ISA server’s external IP address option and type in the IP address of the VIP in the text box. Click Apply and then click OK.

Figure 20

Click on the Local Computer tab in the Allow L2TP protocol packets Properties dialog box (figure 21). Select the This ISA server’s external IP address option and type in the IP address of the VIP in the text box. Click Apply and then click OK.

Figure 21

Click on the Local Computer tab in the Allow L2TP protocol IKE packets Properties dialog box (figure 22). Select the This ISA server’s external IP address option and type in the IP address of the VIP in the text box. Click Apply and then click OK.

Figure 22

The packet filters will take effect in a few moments. You do not need to restart any ISA Server service or the server itself. This may take longer if the server is very busy. You can make the packet filters take place immediately if you restart the firewall service.

The ISA firewall/VPN server array is now ready to accept incoming PPTP and L2TP/IPSec VPN client connections. Incoming requests will be split evenly between all members of the NLB array. If an array member goes offline while a VPN client is connected, the user running the VPN will see the connection fail. When the user reconnects (or when the VPN client software automatically redials), a new VPN connection is established to another member of the array on the same VIP.

Conclusion

With the introduction of the Windows Server 2003 NLB service, we see the realization of the promise of real fail over and load balancing for both PPTP and L2TP/IPSec clients on machines running the ISA Server 2000 firewall software. There were some serious issues that prevented you from taking full advantage of NLB for VPN connections in Windows 2000. Those limitations have been completely removed in Windows Server 2003. I wholeheartedly recommend that you try out a Windows Server 2003 NLB array running ISA firewall/VPN server machines. I think you’ll be impressed!

I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to

http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=13;t=001639#000000 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom