If you would like to read the next part in this article series please go to Load Balancing Web-Proxy Clients With ISA Server 2004 Standard Edition (Part 2).
Load-balancing Web-proxies is generally understood to be one of those things that ISA Server Enterprise Edition does. But for many, the additional cost of the Enterprise Edition might be difficult to justify despite their organisation’s desire for redundancy and load-balancing. Yet the Standard Edition isn’t devoid of tricks to allow load-balancing, and if you intend to have just two or three ISA Servers working efficiently alongside each other, this article may provide the solution you are looking for.
Introduction
With ISA Server 2004 (and 2000 for that matter), if you want to have more than one Web-proxy server in your organisation and have them work effectively together as a single fault-tolerant unit, you go out and purchase the “Enterprise” edition. The Enterprise Edition supports the Cache Array Routing Protocol (or CARP) and that will provide the mechanism to bind your proxies together. But is it really that clear cut? If you have only a few hundred users and must have redundancy in your Web-proxy arrangement, do you really need the substantial extra cost of a couple of Enterprise licences?
Fact is, there is more to CARP than reading about Enterprise Edition might imply, and some aspects of CARP are lurking in the Standard Edition waiting to be let out. For two or three proxy servers in an “array” you may find that you can cost effectively use Standard Edition. Any more than three and it isn’t CARP you need Enterprise Edition for, it’s centralised management and the other “Enterprise” features it supports.
So, say you have decided on two Proxies to provide redundancy. You want them both to share the load and you want to avoid the additional cost and complexity of Enterprise Edition: What are the options?
DNS Round-Robin and Network Load-Balancing (NLB)
Both these mechanisms can be used to provide something like fault-tolerance and load-balancing and I suspect some of you will have tried it. In both cases you configure a common name (and in the case of NLB, a common IP) to point to all your ISA servers. Bear in mind that the more sophisticated NLB is not a Microsoft supported configuration when using Standard Edition.
You might configure your browser to use the common proxy name directly, or you might configure for “automatically detect” or the “automatic configuration script”. These “automatic” options have a distinct advantage over the static method in that you can specify a backup route for when everything goes pear-shaped.
Be aware that if you use the “automatic” methods the browser will download a configuration file from one of the ISA Servers (as decided by NLB or round-robin) and that file will then tell the browser to use only that proxy for the duration. This may not be the load-balancing you might be expecting! To get rapid fail-over using “automatic” methods you can configure settings on each ISA Server to use the other ISA Server as the backup route. ISA Server will then add this information to the configuration file that the browsers receive.
The big drawback of these methods is that each ISA Server builds a cache that will contain information also held on the other ISA Server. This is not an efficient use of resources so this is as far down this path as this article is taking you.
Proxy Automatic Configuration (PAC) Files
If you configure “automatically detect” or “automatic configuration script” in your browser, the configuration file doesn’t have to come from ISA Server; it could be pointed to another location that contains a customised configuration script that you provided and not the ISA Server. This method opens a whole world of possibilities for load-balancing and fault-tolerance.
So what are these configurations scripts? Nearly every browser these days optionally supports the use of automatic configuration scripts. They are written in JavaScript and the browser runs them and then calls a specific function in the script every time it has a URL to fetch. The function returns the Web-proxy that the browser should send the URL request to.
So how do these scripts load-balance? The common method has the script running a “hash” algorithm on the requested URL and using the resultant hash to determine which Web-proxy to send the request to (the algorithm turns the URL string into a unique number called a hash). The great advantage of this method is that each browser runs the same hash function and determines the same Web-proxy for a given URL completely independent of the Web-proxy server. That means the Web-proxies build a cache that is unique to its partner Web-proxies and so provides efficient use of these resources.
Figure 1: A proxy script in action? Err, perhaps not, but I hope you get the idea!
There are numerous hashing algorithms that can be used. Sharp developed the “Super Proxy Script” back in 1996 and you can take a peek at this at http://naragw.sharp.co.jp/sps/. But for most people this technique is associated with CARP.
In this article we will explore customised configuration scripts that utilise CARP. If that sounds scary do not worry, whatever your preconceptions about Enterprise Edition supporting CARP and Standard Edition note, it is ISA Server Standard Edition that is going to do all the scary work for us!
Cache Array Routing Protocol (CARP)
As mentioned above, there is more to CARP than Enterprise Edition sales literature makes obvious. CARP has two implementations; client-side CARP as we’ve already been discussing and server-side CARP.
Server-side CARP
This uses a similar, if not the same, hashing mechanism but this time it’s the server that uses it to determine if it might have the requested URL in its cache or, if not, which of its partners might have it. This is great when the client requesting the URL can’t or won’t support client-side CARP (“Secure NAT” clients in ISA Server-speak) or if the server has upstream proxy arrays and must decide the best proxy to forward the request to.
ISA Server Enterprise Edition supports server-side CARP but you must enable this feature. Standard Edition does not support server-side CARP at all.
Client-side CARP
It is the browser that supports client-side CARP so it matters not if the Web proxies are Enterprise or Standard Edition (or even ISA Servers!). However, Enterprise Edition creates automatic configuration files with all the necessary client-side CARP features in place and it does this even when you have not enabled CARP in the configuration. But so does Standard Edition. Where Standard Edition fails is that it has no understanding of having partners so can’t configure this necessary information in the script it prepares.
Why Microsoft has Standard Edition putting out non-functional client-side CARP code in configuration scripts is anyones’ guess, but it does mean that with a bit of help we can get the script fully functioning.
So let’s take a look at this script ISA Server Standard Edition generates. Just open your browser and type in http://myISAServer:8080/wpad.dat (please use your server’s name not the one here!). Save the file when requested and then open it in a text editor such as Notepad.
Here’s something like what you will see although some lines have inevitably wrapped here. Yikes! I’ve added a bit of colour to make it less scary but the weak of heart might like to close there eyes and skip to the more comprehensible English dialog that follows. But don’t give up as it really is easy in the end.
So what’s going on here? Well your browser, configured with this script’s location, will download it and execute it. This will run the code coloured red that sets up some global variables and runs some of the functions (purple) in the process. From then on whenever the browser needs to request a URL it calls the function FindProxyForURL (in orange) which will return the proxy server to send the URL request to, or else “DIRECT” which means don’t use a Proxy for this URL.
So what’s all that code in blue? The FindProxyForURL function calls these functions to create the hash from the URL and calculate a score for each of the proxies it knows about. The FindProxyForURL function actually returns a list of proxies with the highest scoring proxy first and the configured backup route last, so the browser will use this highest scoring proxy to send the URL request to. That is client-side CARP in action.
Hang-on, but this script came from ISA Server Standard Edition, so what does all this hashing code do? Absolutely nothing! If you look near the beginning there is a function (in purple) called MakeProxies that lists the ISA Server that created this script. There can only be one proxy listed here so the script must always return that proxy. As we all know, Microsoft love using our processors to run totally un-necessary code and in this fine example our poor browsers run this mainly useless code every time it requests a URL, perhaps several times per Web page! This bit of nonsense is going to help us no end.
Creating a Basic Custom Configuration Script
All those that thought this was going to be difficult are in for a shock! Of course the real problem is getting the resulting script out to the browser clients, but we’ll deal with that later. We’ll also deal with some clever edits later too (in Part 2), but for now let’s just get a basic functioning script.
Take a look at the following lines in the script downloaded from ISA Server:
cNodes=1;
function MakeProxies(){
this[0]=new Node(“10.245.10.254”,0,1.000000);
}
The “10.245.10.254” is obviously my ISA Server and you will have the IP of your ISA Server here instead. You might have the fully-qualified domain name, either is good.
Now edit these lines to include your second ISA Server:
cNodes=2;
function MakeProxies(){
this[0]=new Node(“10.245.10.254”,2032180928,1.000000);
this[1]=new Node(“10.245.10.253”,2843172549,1.000000);
}
That’s it!
Okay some mysterious numbers have been added to make the hashing algorithm select one proxy or the other in roughly a 50/50 manner depending on the URL in question. Later I’ll let on where those numbers came from – for now just believe I’m an amazing mathematician!
HTTP Port
Look closely and you’ll find a line similar to:
HttpPort=”8080″;
It tells the browser which port your ISA Servers are listening to for proxy requests on. There is only one such entry, so for this script that means all your nodes must be configured to use the same port. The “8080” port is the default and rarely is it necessary to change this.
Deploying a Custom Configuration Script
There are two mechanisms for getting this script onto your browser clients; either configure them to use “automatically detect” or provide the location of the “automatic configuration script”. Before that you need to publish your script on a suitable Web site that your browser has access to.
If you’re going to have redundant proxies you will need redundancy for the Web site where you are publishing your configuration file or there will still be a single point of failure. To illustrate the required configuration I’ll step through the creation of a single IIS Web-site, but you may well use your established fault-tolerant Intranet site or any other suitable arrangement.
For this illustration we’ll start by creating a new site in IIS Manager:
Figure 2
Next step in the Web Site Creation Wizard is to give the site a descriptive name. I’ve used what WPAD actually stands for as you can’t get more descriptive than that.
Figure 3
For the next page you could specify an IP (or leave it as it is here) and I would recommend sticking with port 80 because there is an auto-discovery feature that needs this port (we’ll discuss that in Part 2). A host header will probably be necessary because port 80 may be shared by other virtual servers.
Figure 4
Remember to have your DNS resolve whatever address is used for the host-header. You may have two entries pointing to two Web sites (DNS round-robin), or point the record to a network load-balanced virtual IP, or else use whatever technique your fault-tolerant Web-site employs.
Following that, you need to provide a location for the file and note that it is essential that this site allows anonymous access. The next Wizard page following this one is for access permissions; for this site you only require Read permission (no script, execute or write permissions).
Figure 5
Finishing the Wizard will complete the essential configuration of a suitable Web-site. It is then just a matter of copying your configuration script to the appropriate location (C:\Inetpub\wpadroot was used here) and ensure the anonymous access account (usually IUSR_Servername) has read access to this file location.
Figure 6
Check all is well by browsing to the file and downloading it (in this example the URL would be http://wpad.company1.local/wpad.dat).
I named the configuration script WPAD.DAT, though there is nothing demanding that it must be called so.
Configuring Internet Explorer to Use Configuration Scripts
Manually configuring Internet Explorer to use a configuration file is straight-forward. Start by selecting Internet Options from the Tools menu.
We need the Lan Settings option found on the Connections page.
Figure 7
Enter the URL that locates the WPAD.DAT file you’ve published and click OK.
Figure 8
That’s finished. Browse to an Internet Web-site and the browser will start using the configuration script to pick the proxy to send the request through. Like so:
Figure 9
Ahh… perhaps I should have configured my ISA Server rules! But, nothing proves you’re going via an ISA Server more than this little page.
Of course, manually configuring browsers is not what you want if you have 100s of clients to deal with. Group Policy is a good way of configuring a multitude of clients but I’ll leave the discussion about that until Part 2. Also in Part 2 we’ll look at that other automatic configuration option, Automatically detect settings, but be warned that there are arguments against this approach.
Conclusion
In this article we’ve edited the configuration script supplied by ISA Server 2004 Standard Edition and deployed it to our browser clients. As a result, two or more ISA Server Standard Edition Web-proxies can be efficiently load-balanced using client-side CARP. The loss of an ISA Server proxy in this arrangement will quickly be covered by its partner.
The information here is all you need to get up and running, but in Part 2 of this article we’ll look at some more advanced edits of the custom script. I’ll illustrate some manual changes to some other features in the script because ISA Server will not be doing them for you any more. We will also introduce the concept of “CARP Exceptions”, a feature in Enterprise Edition to avoid problems with some Web sites.
But before we finish here …
Calculating Hash Values for Proxy Nodes
If you remember, there were two magic numbers entered in the script that the hashing algorithm would use to ensure each ISA Server gets a fair share of the load. The numbers were 2032180928 and 2843172549. I’ll have to come clean and admit I have no idea how these numbers are calculated!
What I did was install the 120-day evaluation version of ISA Server 2004 Enterprise Edition on some virtual machines and check the values in a WPAD.DAT file downloaded from one of the servers. Yes it is cheating, but very effective!
You could use the same trick to explore other values in the script but whatever you do don’t install ISA Server 2004 Service Pack 2: SP2 introduced changes to Enterprise Edition that includes an improved hashing algorithm in the configuration script. So unless you want to get your head around those changes, perhaps running foul of copyright by using them, stick to pre-SP2.
By the way, the magic number for a third ISA Server in an array is 3804533832 and the other numbers stay the same.
If you would like to read the next part in this article series please go to Load Balancing Web-Proxy Clients With ISA Server 2004 Standard Edition (Part 2).
