If you missed the first part in this article series please read Load Balancing Web-Proxy Clients With ISA Server 2004 Standard Edition (Part 1).
Part 1 illustrated how it is possible to load-balance two ISA Server Standard edition Web-proxies with just a little tampering with the proxy automatic configuration file that it creates. For many of us this trick would avoid the additional cost of the Enterprise edition. In this Part 2 of the article we’ll tamper a little more to extend the load-balancing capabilities further still.
Introduction
In Part 1 we took the “wpad.dat” file output by one of a pair of ISA Server Standard Edition servers and slightly modified a few lines in the file. This started as:
cNodes=1;
function MakeProxies(){
this[0]=new Node(“10.245.10.254”,0,1.000000);
}
Was changed to become:
cNodes=2;
function MakeProxies(){
this[0]=new Node(“10.245.10.253”,2032180928,1.000000);
this[1]=new Node(“10.245.10.254”,2843172549,1.000000);
}
By making the resultant file available through a Web-site to browsers such as Internet Explorer, this change enabled the routines lurking within the file, and as a result two servers running Server Standard Edition become load-balanced and fault-tolerant for Web proxy services.
However, this simple action does mean configurations made to ISA Server do not make it to our now orphaned file: A few more manual edits to our file are in order to make it properly useful.
Part 1 also illustrated how to manually configure Internet Explorer to pick up the new configuration file: This was not a practical solution for most of us so to start with we’ll cover the Internet Explorer Maintenance add-on in Group Policy to do this job for us.
Group Policy and Internet Explorer Maintenance
Before diving into Group Policy, the ISA Server built-in mechanism for managing IE in the Firewall Client should be mentioned: The reason I’m skipping this mechanism is firstly that many ISA Server deployments are as Web proxies only (sadly) and the Firewall Client isn’t deployed; secondly I have a preference for Group Policy anyway, and thirdly, we’ve all seen the method of using Firewall Client in various other articles! So, back to Group Policy.
The Internet Explorer Maintenance add-on in Group Policy isn’t a favourite feature of mine. It just doesn’t work like the rest of the settings in Group Policy and some of the settings appear duplicated in the “Administrative Templates” section. The Internet Explorer Maintenance add-on does much of what IEAK will do but in a Group Policy; in fact it is the client-side IEAK components that this add-on is utilising to apply the settings.
However, it does do the job so let us run through the requirements.
Configuring Automatic Browser Configuration
First is to select the Group Policy to edit. You wouldn’t want to pollute the Default Domain Policy with these settings so a separate policy is in order.
Open this policy up and locate Internet Explorer Maintenance from the User Configuration, Windows Settings node. Right-click it to get the menus.
Figure 1
We need Policy Mode which really means not Preference Mode, so don’t click it and ensure there’s no tick against it. Once you make a change to IE Maintenance you’ll notice you just can’t clear these changes from the Group Policy; to do that you need that Reset Browser Settings option (greyed out here because no settings have been made yet). Expand the IE Maintenance mode with a double-click.
Figure 2
Under Connection you’ll find Automatic Browser Configuration; right-click this to select Properties.
Figure 3
Check Enable Automatic Configuration and enter your Auto-proxy URL. Notice it asks for JS, JVS or PAC files but the example being used here is a DAT file; your browser will not care!
Click OK and you are done: At least you would be forgiven for thinking you are done but that is not how this IE Maintenance add-on works. If you go into Internet Explorer and bring up LAN Settings (Tools, Internet Options, Connections tab) you will indeed see that the Group Policy has added the new settings (if you’ve waited for the policy to apply or else run GPUPDATE.EXE), but you can still untick the setting again! That is not what you want for your users.
Things get worse: you’ve unticked the setting, now wait for Group Policy to apply again (or go for GPUPDATE) and you’ll wait in vain for the setting to come back again. IE Maintenance will not apply the setting again unless something in the policy is changed. This behaviour can be very confusing; Microsoft obviously had something else in mind when describing “Policy Mode”.
Enforcing the Policy
To put this right, look further down the “User Configuration” settings of the Group Policy to the Windows Components node of Administrative Templates.
Figure 4
Under Internet Explorer you’ll find two settings: Disable changing proxy settings and Disable changing Automatic Configuration settings. Enable both of these (or at least the latter).
You can also drill down further to Internet Control Panel and enable Disable the Connections page to make the entire tab disappear. This setting is also available in “Computer Configuration” which has priority too.
But even after this much work your settings aren’t safe from meddling by other means such as registry editors, and remember that your policy won’t apply again until the policy changes.
Really Enforcing the Policy!
To put this right requires a change to the “Computer Configuration” settings of a Group Policy. Drill down through Administrative Templates, System and Group Policy to find Internet Explorer Maintenance policy processing.
Figure 5
Double-click this to open it.
Figure 6
You need to at least select Process even if the Group Policy objects have not changed, and probably also Do not apply during periodic background processing.
There is a sort of reason why this isn’t the default behaviour: If you use IE Maintenance for some of the extensive branding it allows, you only want changes to occur once and not every logon, or worse, every 90 minutes or so with the rest of Group Policy. With this in mind think carefully about how you use IE Maintenance before blindly following this step.
Automatically Detect Settings Option
You’ll have noticed the Automatically detect settings option alongside the Use automatic configuration script option used throughout this article. You may use this alternative if you want, but there are pitfalls which is why I’ve been avoiding it so far.
If you followed Part 1 closely you’ve already created a Web site with a name similar to the “wpad.company1.tld” used in the example. The important bits are that the site is called “WPAD”, that it responds on port 80 and that the file is called “wpad.dat” (as recommended in Part 1). If that’s the case, select Automatically detect settings and all should work; your browser will go off looking for http://wpad/wpad.dat and it should find it!
The other way of supporting Automatically detect settings is to have an entry in DHCP. This “Option 252” entry has a format like http://wpad.company1.tld:80/wpad.dat. The advantages here being that you can call the file (and path to it) whatever you like and use whatever TCP-port for the Web-site you like too. But here come the drawbacks:
The browser will make a DHCP request for this value, but before Windows XP SP2 and 2003 SP1, only administrators were allowed to do that: Some of your clients may not work with this!
And: Whether or not your browser is allowed to locate the DHCP entry and successfully, or not, download the script, it will sit around for some time just thinking about it! This presently requires a hotfix or two from Microsoft to put right.
For an in-depth (and I mean depth!) discussion on matters of the Automatically detect settings option see Stefaan Pouseele’s excellent article (with links to updates and hotfixes).
For now, I’m perfectly happy recommending the Use automatic configuration script option.
Exceptions
ISA Server does a fine job of fetching Web content from external networks, but the last thing most of us want is for your browser to forward requests to the proxies for Web pages hosted on your internal intranet servers. For this content the browser wants to go direct to the Web site.
ISA Server allows you to configure these “exceptions” and it will take these entries and place them in the automatic configuration (PAC) file it creates; that’s the same file we’ve been tampering with. When you download a “wpad.dat” file to follow this article you may see some of these exceptions. The following example lines from the beginning of the PAC file has had no such exceptions configured yet:
//Copyright (c) 1997-2004 Microsoft Corporation
BackupRoute=”DIRECT”;
UseDirectForLocal=true;
function MakeIPs(){
}
DirectIPs=new MakeIPs();
cDirectIPs=0;
function MakeNames(){
}
DirectNames=new MakeNames();
cDirectNames=0;
Our PAC file is now orphaned from ISA Server and must be manually edited, but exceptions can be accommodated by editing just these few lines.
BackupRoute
This isn’t really an exception, just a “what to do if all else fails”. If your firewall (which might also be your ISA Server) still allows direct access to the Internet for Web content, or could be quickly reconfigured to do so in an emergency, then “DIRECT” is the best option here.
You might alternatively configure another Web proxy that could take the load in an emergency. The format of the line to allow this will be something like:
BackupRoute=”PROXY W3PROXY.COMPANY1.LOCAL:8080“;
You would of course replace the name here with your own (or an IP address) together with the port used. As the whole point of this article is to pair ISA Servers to provide fault-tolerance, we’d hope this backup route is never necessary!
UseDirectForLocal
You almost certainly want this to be “true” and it may even be all the “exceptions” you need. It will mean that any URL where the fully-qualified domain name, or host name, has no “dots” go direct and not to the ISA Server. So http://www.isaserver.org/pages/newsletters.asp goes through ISA server but http://myserver/intranet/index.html will go direct.
DirectIPs
If you use IP addresses to browse your Intranet, or have links in your Intranet pages that reference IPs, then you will want to include an edit here. An example might be:
function MakeIPs(){
this[0]=”172.16.0.0″;
this[1]=”255.240.0.0″;
}
DirectIPs=new MakeIPs();
cDirectIPs=2;
The important thing to note here is that the entries are in pairs, a subnet and a mask, and that the “cDirectIPs” value reflects both entries, so it is 2 here yet this edit only provides one subnet entry. You can add further pairs; just keep incrementing the “this[n]” lines and that “cDirectIPs” value.
This is quite ugly code, but you won’t have to edit it much and it probably doesn’t justify cleaning up the code to make it easier to edit (but you are free to try!).
DirectNames
This works in a similar manner to “DirectIPs” but here you list all those host names that you don’t want to go through the ISA Server.
function MakeNames(){
this[0]=”intranet.company1.com”;
this[1]=”*.local”;
this[2]=”*.develservers.tld”;
}
DirectNames=new MakeNames();
cDirectNames=3;
Note you can specify wildcards to include an entire domain, but only use the wildcards at the beginning of the name; “www.company1.*” doesn’t work!
CARP Exceptions
Because the PAC file is used by the browser for every URL request, it follows that the browser will use more than one proxy for various pages on the same Web site. Some Web sites will get terribly upset when they get requests coming from the same client yet from two different IP addresses.
This isn’t a problem if your ISA Servers are behind a “NATing” device (firewall), but could be if your ISA Server Web-proxies are also your perimeter firewalls with public IP interfaces.
To avoid problems with these Web sites the browser must be instructed to send all requests for a specified host through the same proxy. In ISA Server Enterprise Edition this is described as “CARP exceptions”, and to get the same effect from our PAC file is going to require a little additional code.
The first step is to add the next few lines near the top of the file just after the line that gives “cDirectNames” its value:
cDirectNames=0;
function MakeCARPExceptions(){
}
CARPExceptions=new MakeCARPExceptions();
cCARPExceptions=0;
The next step replaces the FindProxyForURL function with these lines (changes are in red):
function FindProxyForURL(url, host){
var urlhash, urllower, ibest, bestscore, list, i, j, port=HttpPort, hostonly=false;
urllower = url.toLowerCase();
if((urllower.substring(0,5)==”rtsp:”) ||
(urllower.substring(0,6)==”rtspt:”) ||
(urllower.substring(0,6)==”rtspu:”) ||
(urllower.substring(0,4)==”mms:”) ||
(urllower.substring(0,5)==”mmst:”) ||
(urllower.substring(0,5)==”mmsu:”))
return “DIRECT”;
if (UseDirectForLocal && isPlainHostName(host))
return “DIRECT”;
if (cDirectNames > 0)
for (i = 0; i < cDirectNames; i++)
if (shExpMatch(host, DirectNames[i]))
return “DIRECT”;
if (cDirectIPs > 0)
for (i = 0; i < cDirectIPs; i += 2)
if (isInNet(host, DirectIPs[i], DirectIPs[i+1]))
return “DIRECT”;
if (cCARPExceptions > 0)
for (i = 0; i < cCARPExceptions; i++)
if (shExpMatch(host, CARPExceptions[i])) {
hostonly = true;
break;
}
urlhash = HashString(url, hostonly);
for (i = 0; i < cNodes; i++)
Proxies[i].score = Proxies[i].load * Scramble(MakeInt(urlhash ^ Proxies[i].hash));
list = “”;
for (j = 0; j < cNodes; j++) {
for (bestscore = -1, i = 0; i < cNodes; i++) {
if (Proxies[i].score > bestscore) {
bestscore = Proxies[i].score;
ibest = i;
}
}
Proxies[ibest].score = -1;
list = list + “PROXY ” + Proxies[ibest].name + “:” + port + “; “;
}
list = list + BackupRoute;
return list;
}
Finally replace the entire function with this modified one:
function HashString(url, hostonly){
var h = 0;
var slashes = 0;
for (var i = 0; i < url.length; i++) {
var c = url.charAt(i);
if (c == ‘/’)
slashes++;
if (slashes < 3) {
c = c.toLowerCase();
} else if (hostonly) {
i = url.length;
}
h += (((h & 0x1fff) << 19) | ((h >> 13) & 0x7ffff)) + CharToAscii(c);
h = MakeInt(h);
}
return h;
}
Activate the code in a similar manner to “MakeNames” but listing all those host names that you want to go through the same ISA Server:
function MakeCARPExceptions(){
this[0]=”crabby.website.tld”;
this[1]=”gripping.tld”;
}
CARPExceptions=new MakeCARPExceptions();
cCARPExceptions=2;
These modifications have been influenced by the PAC file ISA Server Enterprise Edition produces; it is functionally the same but not the same code because firstly there might be copyright issues, and secondly I didn’t want the huge amount of additional lines that is added to the Enterprise Edition version!
Variations (for the Experimentally Minded)
There is another option to the above CARP exception changes: Instead just replace the HashString function with:
function HashString(url){
var h = 0;
var slashes = 0;
for (var i = 0; (i < url.length) && (slashes < 3); i++) {
var c = url.charAt(i);
if (c == ‘/’)
slashes++;
h += (((h & 0x1fff) << 19) | ((h >> 13) & 0x7ffff)) + CharToAscii(c);
h = MakeInt(h);
}
return h;
}
Change the line that calls this function in FindProxyForURL to:
urlhash = HashString(urllower);
In this case all requests for pages on a particular hostname go through the same proxy (i.e. all requests are “CARP Exceptions”). I’m not recommending this approach as I have no idea on how much it would effect load-balancing, but you might like to experiment and save your browser from a bit of processing too.
There are plenty of other opportunities to mess with the PAC file ISA Server generates and come up with something much tighter: If that’s what you enjoy, go ahead!
What is Client-side CARP?
A small digression: I’m somewhat uncomfortable with the term “client-side CARP”. The “P” in CARP stands for “protocol” so just what makes a Proxy Automatic Configuration file become a protocol? Still, this seems to be an accepted terminology so I’ve gone along with it. Apologies if you too are confused, either before or after you read this box!
Conclusion
ISA Server 2004 Enterprise Edition is generally the product of choice for load-balancing Web proxies using ISA Server. But these articles have shown that, if you can accept the minor inconvenience of disassociating the PAC file from ISA Server, it is fairly simple to introduce effective load-balancing and fault-tolerance to ISA Server 2004 Standard Edition.
If you missed the first part in this article series please read Load Balancing Web-Proxy Clients With ISA Server 2004 Standard Edition (Part 1).
