Using NLB with ISA Server Part 3: Configuring NLB Array Parameters
By Thomas W Shinder M.D.
In the first two parts of this three part article on the basics of NLB we went over basic NLB features such as convergence, the NLB algorithm, multicast/unicast modes, and a lot more. In this last article in the series we’ll look at the Windows 2000 NLB configuration interface and explain what the options are and what they mean. Next week we’ll take the information you learned about NLB and apply to it scenarios that employ NLB for providing load balancing and fault tolerance for VPNs, Server Publishing, Web Publishing and outbound access for internal network clients.
In this article we’ll cover of the following topics:
Configuring Array Parameters
When configuring an NLB array, you need to set parameters for the entire array and parameters for individual members of the NLB array. The first NLB dialog box encounter is the Array Parameters tab. Let’s go over the meaning of each of the options on this tab.
Primary IP address – The is the Primary IP address used by the array hosts. All hosts must be configured with this address. You can configure multiple virtual IP addresses, but you need to enter a primary IP address that all array hosts share. You enter this IP address on all array members. You need to bind this address to the interface using the Advanced TCP/IP Properties dialog box. Just make sure this isn’t the top listed IP address in the list.
NOTE:
In the ISA Server vocabulary, we usually refer to the “primary IP address” as the one on the top of the list of IP addresses bound to the external network interface. This is not the case in the NLB vocabulary, where the Primary IP address is just one of the IP addresses used by NLB (although the array Primary IP address is the one you use if you want to remote control the array). What’s important is the distinction between the NLB Primary IP address and the array’s Dedicated IP address. We’ll get to the Dedicated IP address in a little bit.
Subnet Mask – This is the subnet mask for the primary IP address and all other virtual IP addresses bound to this network interface. Keep in mind that the Windows 2000 NLB can be bound to on a single interface, so you have to choose between the internal or external interface on the ISA Server. Although it should be obvious, be aware that all array members need to be on the same network ID. You can’t use NLB to provide fault tolerance for ISP connections. You can’t get multiple ISPs to assign you IP addresses on the same network ID. If you want fault tolerance and load balancing for your ISPs, then check out RainFinity’s RainConnect [www.rainfinity.com] application.
Full Internet name – This is the FQDN that resolves to the Primary IP address listed in this dialog box. Its not clear from the documentation that I’ve read whether or not this has any actual relevance. I haven’t noticed any, but I could be missing something. It might just serve as a reminder that if you want to remote control using a FQDN, then you need to make sure the FQDN resolves to the Primary IP address on the NLB array. If you know if what you enter here makes any different, let me know at [email protected].
Network Address – This is the MAC address assigned to the array. If the address begins in 03, then you’re using multicast mode. If the address begins in 02, then you’re using unicast mode.
Remote password/Confirm password/Remote Control – These are passwords used for remote management of the array. If you want to enable command line remote control of the NLB array, you can put a checkmark in the Remote control checkbox and enter the passwords. We won’t be going over issues of remote control in our ISA Server centric discussions. If you want to know more about your remote control options, type wlbs /? At the command prompt.
Configuring Host Parameters
Host parameters apply to a specific member in the NLB array. Click on the Host Parameters tab and you’ll see what appears in the figure below.
Priority (Unique host ID) – This value is used for a number of things. As you saw in part 2 of this series, this value is used to determine the value of an octet in the bogus MAC address that’s used to trick the switch into allowing all array hosts to use the same MAC address when you choose unicast mode. The Priority value is used to identify each host in the array; this is how the NLB algorithm recognizes each array member and assigns client connections based on members’ host ID. The host ID is also used for fail over for connections that don’t have an explicit port rule assign to them.
For example, you might have a port rule for the array that applies to TCP 80 for your incoming Web requests. The array has a VIP of 2.2.2.2. You also publish an SMTP server and configure your MX records to point to 2.2.2.2. While your incoming HTTP connections will be load balanced, your SMTP messages won’t be load balanced (because you need to configure port rules for load balancing), but there will be fail over for incoming SMTP connecitons. Host ID 1 will receive all the incoming SMTP connections. If Host ID 1 fails, new incoming SMTP connection requests are forwarded to host ID 2. If both host IDs 1 and 2 fail, then connections will be forwarded to host ID 3, and so on.
Initial array state – This value allows you to have NLB started automatically. You will have to start the service manually if you don’t select this value. You’ll always want to enable this value in your ISA Server specific configuration of NLB
Dedicated IP address/Subnet mask – The dedicated IP address is an IP address that is specific for each member of the NLB array. No two members of the array should have the same dedicated IP address. Connections to this IP address are never load balanced and never failed over. You use this IP address to communicate with the specific array member.
If you are using NLB on the external interface of the ISA Server, you should list the ISA Server’s primary IP address as the dedicated IP address. This will cause the source IP address of all outbound connections to be the dedicated IP address. This is important because the NLB algorithm may cause responses to return to an array member that is not the array member that sent the request. Keep in mind that the array members do not share “state” information with one another – they share neither connection state nor application state. That is to say, none of the ISA Servers are aware of connections made by other ISA Servers in the NLB array.
For example, suppose you have two NLB array members: ISA1 and ISA2. They have the following IP addresses:
ISA1:
Dedicated 1.1.1.2
VIP 1.1.1.1
ISA2:
Dedicated 1.1.1.3
VIP 1.1.1.1
Now suppose you configured the VIP as the ISA Server’s primary IP address (the one on the top of the list of IP addresses on the external interfaces of the ISA Servers). You have a client on the internal network that sends an outbound PORT mode FTP request to an FTP server on the Internet. This outbound request goes through ISA1 with a source IP address of 1.1.1.1 (since ISA Servers always use the primary IP address as the source address for outbound messages). The FTP server on the Internet returns the data from its own TCP port 20 to IP address 1.1.1.1. The NLB algorithm then assigns the incoming request to ISA2. This causes the FTP session to fail because ISA2 is unaware of the FTP session that was established by ISA1.
Is there a solution to this problem? Yes, but you have to be aware of how port rules affect load balancing. What if you don’t have a port rule that affects TCP 20? Then all incoming requests to that port on the NLB VIP will be directed to the server with highest Host Priority. So, if ISA1 has the highest Host Priority, then all incoming messages for ports that don’t have port rules defined will be directed ISA1. This gets you around the NLB algorithm assigning the response to ISA2.
But is this a real solution? It depends on how you want to use your array. Remember that ISA Server on Windows 2000 does not support bi-directional affinity.
It’s likely that some internal clients are using ISA1 for outbound access and some internal clients are using ISA2 for outbound access. This reintroduces the problem of incoming responses being distributed to the wrong server. When an internal network client sends an Internet bound request to ISA2, the request is forwarded to the Internet server from ISA2 with the source address 1.1.1.1. Let’s say that ISA2 used TCP port 25550 for the source port on the outbound message. The Internet server then sends the response to the VIP 1.1.1.1 on TCP port 25550. There is no port rule assigned to TCP port 25550 and the NLB array members do not share connection state information. Since there is no port rule and no connection state sharing among the array members, the response to 1.1.1.1 TCP port 25550 will be send to the host with the highest Priority (lowest host ID number).
If NLB shared connection state information among all the array members, then a shared state table would have identified which host ID made the outbound request and there would be no problem. If fact, you don’t even need a shared connection state table; you could use hashing algorithms that applied to both internal and external connections.
Unfortunately, Windows 2000 NLB does not support bi-directional affinity. The ISA Server is happy to accept responses on the VIP, but NLB has no influence over, or awareness of, the firewall’s state table. ISA2 has a state table entry allowing it to expect a response from the server it sent the outgoing connection request to. ISA1 does not have a state table entry for this connection. When the Internet server sends its response to the VIP and its assigned to ISA1, the request will be dropped as an unsolicited inbound packet.
The figure below shows what happens when an internal network client configure to use ISA2 sends an outbound request for a Web page to a server located at 222.222.222.1 and the ISA Server primary IP address is a VIP. NLB forwards the response to ISA1 and the ISA1 drops the packet.
- The internal network client sends an outbound request to a Web server on the Internet. ISA2 intercepts the request and changes the source IP address of the outbound request to 1.1.1.1 and the source port number to TCP 25550.
- The Web server returns its response to IP address 1.1.1.1 and port number TCP 25550, since that was the source information contained in the request forwarded by the ISA2.
- NLB intercepts the incoming response to 1.1.1.1 and does not load balance the response to ISA2.
- NLB does load balance the response to ISA1. Unfortunately ISA1 is not aware of the connection request and drops the response, as it considers the response as unsolicited.
The bottom line here is to make sure that you always use the Dedicated IP address the ISA Server’s primary IP address.
Port Rules
The Port Rules tab allows you to configure rules to control how incoming packets are load balanced. Notice that there is a rule that applies to all TCP and UDP ports. This is no problem if you are using multicast mode and you do not configure an NLB address to be the ISA Server’s primary IP address.
Port range – This allows you to configure a range of ports, or even a single port, to which the is applied. Note that the existing rule already appears in the bottom frame of the dialog box. You are not editing the existing rule when you start working in this dialog box; you are creating a new rule and at the end you have to click the Add button.
Protocols – You can choose which transport layer protocol you want the port rule to apply to. This allows you to fine tune your rule.
Affinity – determines how client/server “stickiness” is determined, as we covered earlier in our discussion on client affinity.
Load weight – you can customize how much of the load is assigned to this server. The default is to assign incoming packets equally to all members of the array. If one of your array members is “beefier” than others, then you might want to assign a higher load weight to that server. That will cause the server to be assigned a proportionally higher percentage of the incoming packets. You can enter any number between 0 and 100 here. Note that this does not represent a percentage, it represents a proportion.
Filtering Mode
The Filtering Mode defines how the array handles the ports denoted in the rule. The default (Multiple Hosts) allows all machines in the array to handle the rule, and load balancing it based on your Affinity and Load Weight settings. The Single Filter Mode allows the port rule to override the default fail over settings and use the settings you put in the Handling Priority box. This allows you to configure fail over for the ports included in the rule, but not load balancing. Each array member needs to be assigned a different Handling Priority that overrides the Host ID for the port or ports included in the rule. The Disabled filtering mode causes all members of the array to drop requests for the ports included in the rule. This allows you to implement another form of packet filtering for the array.
Summary
In this two part overview of the Windows 2000 Network Load Balancing service we went over some of the key concepts in NLB. You learned about basic NLB principles including load balancing, fault tolerance, convergence, multicast and unicast mode, heartbeats, affinity, and configuration parameters. Throughout the articles I focused on issues that apply to our ISA Servers being configured as an NLB array. If you’re going to implement an NLB array, I highly recommend that you review Microsoft more in depth material after reading these articles. The following articles will provide you a lot of important details that fill complete enhance your NLB understanding:
Chapter 19 – Network Load Balancing – Windows 2000 Resource Kit
Appendix B – Network Load Balancing Technical Overview – Application Center Resource Kit
Windows 2000 Server Training – Network Load Balancing – TechNet CD-ROM
I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=2;t=007757 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom
