Designing An ISA Server Solution on a Complex Network.

Figure 1

ISA versions

  • Enterprise Edition  – This version requires a W2K AD-domain to function properly.  Installing Enterprise edition in any other environment reverts the installer to Standalone installer mode, wasting your hard spent $$.

  • Standalone Edition – Standalone Edition is totally ignorant of other ISA servers and ISA arrays.  This is the version to use for non-AD or non-array environments.

Windows 2000 Networking Services
To support your ISA Server and both internal and external ISA Server clients, depending on your requirements, you should have some of the following Windows 2000 networking services installed and configured on your network:

  • DNSrequired – W2K uses DNS before any other name resolution scheme, so installing and configuring it correctly makes name resolution faster.  Generally, if you build one, build another for redundancy.

  • DHCPhighly recommended – The service that makes client IP assignment consistent across all the internal machines, making your life a little easier.  If configured correctly, DHCP services can also help you keep your W2K DNS server updated for non-W2K clients.  DO NOT use DHCP for the ISA server itself, unless you want to take insanity out for a test drive. 

  • WINSoptional – This is a NetBIOS name resolution service that is not specifically required for W2K.  Even if you have a mixed-OS network, as long as internal DNS and client IP options are configured and functioning properly, you can skip WINS (Keep It Simple, Sam).

  • Domain Controller (Active Directory or NT SAM) – situational – ISA arrays (Enterprise Edition) are only truly functional in a W2K AD-domain.  ISA operates in standalone mode in any other environment.

These supporting services can be critical to the proper functioning of your ISA Server network infrastructure.  Standalone ISAs will function in an AD environment, just not as array members.


Overall Description

  • Topology – Figure 1 depicts a summarized version of a “corporate” deployment; one where multiple subnets and network infrastructure devices (switches, routers) are used to connect a wide range of clients and servers.  Since this builds on a previous discussion of a “simple network”, we won’t rehash those topics.

    • Networking devices – All of these devices require IP addresses to facilitate remote configuration and control of these devices.  This presents an interesting security issue, since users can potentially change these settings unless proper limitations are present. Since these devices operate on the network as active TCP/IP devices (with IP addresses and everything), they change the methods required for making client-to-ISA connections.

      • Switches – These are devices that “direct” packets passed on the destination MAC address (layer 2).  While there are devices that are much smarter than that; this is their basic purpose.  Generally speaking, they don’t change the packet contents as they direct them.
      • Routers – These devices “route” packets based on the destination IP address (layer 3) and will change packet data according to rules defined in their settings and as defined in RFCs governing their behavior (the laws of TCP/IP).  Since they change packet addressing, this is where the “default gateway” issue comes into play for SNAT clients.  One important point with routers is that each defined interface uses an IP address.  This IP address will be the “default gateway” for any device connected to it.

        Note: the term “router” is actually inappropriate with regard to xDSL and cable modem sharing devices.  The name “router” is nothing more than a marketing device to make them sound more capable than they really are.  These are actually “switches” or even “hubs” that may provide minimal NAT, DNS and DHCP services.  They are not routers in the strict sense of the term, as they don’t support routing protocols, such as RIP and OSPF.

    • Servers – These machines must be SNAT clients; that is, they must have a default route to the ISA server through the networking infrastructure.  Since they no longer have a direct connection to the ISA server, the “default gateway” method is not viable

    • Workstations – These machines fall into a slightly different security context than do the server machines, since real people are causing them to do their work.  This may necessitate special IP settings for these machines, depending on the role they play in your deployment.

  • Multiple subnets – TCP/IP sub netting is a useful administrative tool that provides for more granular control of the network.  Unfortunately, this also means that the “default gateway” setting has no direct bearing on a client’s ability to connect to the ISA server.  Think of network interfaces as doors in a house: “Default Routes” and “Default Gateways” define the “path of last resort” for a TCP/IP device.  If the destination address is not on the front street or the back street, then we’ll always send it out the “default door” and hope it finds its own way.

    The way SNAT clients are defined is slightly different in this scenario:

    • The client machine must have its closest router defined as its “default gateway”.
    • A “default route” is defined in the router closest to the ISA server (or array) that points to the ISA server internal IP address.  Note that if the ISA server has multiple internal IP addresses, this must be the “default”, or primary IP address for the internal interface.
    • If other routers are used in the network, they must have their default route defined as the next nearest router to the ISA-serving router.  This can get quite involved in complex deployments.
    •  The ISA server must have a “classless” persistent route to all subnets.  What this means is that the ISA server should understand how to reach any subnet in the topology.  If this is not defined on the ISA server, all the “defaults” in the world will no allow client connectivity to the ISA.  A “classless” route is one that encompasses all subnets.  This is accomplished in figure 1 by assigning a route to the “” subnet using a larger-than-defined subnet mask of “”.  The effect of this is to instruct the ISA TCP/IP stack to use the nearest router as the default route for any IP address using “192.168” as the first two octets, regardless of the actual destination IP address.  This is much simpler than entering each subnet and its related subnet mask as each is created.

Figure 2 illustrates this concept.  Each switch shows that all machines connected to it use a default gateway (DG) of the nearest router.  Each router in turn, uses the next router up the chain as its default route (DR), except for the last one which uses the ISA as its default route.  Note that the switch that has the ISA server connected to it uses “RTR3” as its default gateway.  While this might seem to create a potential loop, remember that the ISA server is different than any other server in this subnet; it has its default gateway set to its external interface.  This way, the “unknown destination” for ISA will always use the external interface.


Figure 2

  • Determining the ISA “classless” route – some TCP/IP binary math understanding is required here, as the subnet mask is the key to “classless” routing.  The subnet mask is binary comparison data.  It progresses in binary fashion (1, 2, 4, 8) instead of linear progression (1, 2, 3, 4).  While the number 252 is valid in netmask terms, 253 is not, as it leaves a “hole” in the bit mask.  “Classless” is a term that defines a network as being logically larger than it physically is.  If you define a network as, you’ve created a network that has 254 hosts on 256 subnets.  If you define your route as, then you force the TCP/IP stack to view that network as a single logical subnet with 65534 hosts, all seen through the same interface.

  • Setting the “classless” route – supporting the IP layout in figure 1, the command would be:
    route -p add mask
    – the “-p” makes the entry “persistent”, or permanent.  Manually added routes are not “persisted” between reboots unless the “-p” is in place.  Needless to say, this can cause quite a bit of confusion.  The placement of the “-p” flag is important; type the command as shown or it won’t be accepted.

  • DNS configuration – on the internal DNS server(s), make a “copy” of your external zone (not a secondary zone), but use the internal IP addresses of the published servers.  This will allow the internal clients to resolve external names as internal IP addresses, avoiding the “failed to create packet filter” errors so common with this type of request.  Note that you must have two physically different DNS server in order to accomplish this.  One method is to place your “real” external zones on a DNS installation on the ISA server.

  • Securing Firewall clients – Sometimes we want total control of network traffic.  We may not want the clients to make direct connections to the ISA server without authentication.

    •    Router configuration – First, the routers should not have default routes to any ISA server.  They must have routes to each other to allow cross-subnet traffic, but none need have access to the ISA.  We would configure the routers to “blackhole”, or drop any packet not destined for our local subnets.  This prevents connection to the Internet for clients not using the firewall client.

    • ISA configuration

      • Make all protocol rules “client-specific”; that is, use user / group restrictions that apply according to the user’s needs.  The VP of a group would potentially have greater Internet access than would an intern
      • Make sure you enter the internal domain in the LDT; this is critical to proper routing for firewall client name resolution.

    • ISA client software configuration – Set the auto configuration data as appropriate for your deployment; this is spelled out quite well in the ISA help file.  Pay particular attention to the “WPAD” DNS and DHCP settings, as they are critical to firewall client functionality.  Verify these settings before installing the software on any client.

    • Client configuration – In order to force workstations to use the ISA as their only path through the network to the Internet, we have to install the Firewall client software from the ISA server.  In this case, the client retains its default gateway setting.  This is critical to network connectivity in a multiple-subnet scenario.

    • Server Applications – DNS, SMTP, POP, WEB and similar applications typically don’t respond well to Winsock proxy software, as they’re not Winsock applications in the literal sense of the term.  They prefer to have direct connectivity to the TCP/IP stack and cause much hate and discontent when Winsock proxies are used.  Consequently, you should plan on placing your server applications in the same subnet as the ISA server so that you can make them SNAT clients.  Since we’ve set the routers to drop any packets with non-internal destination addresses, we have to set the server’s default gateways as the internal IP of the ISA server.  This won’t interfere with internal communications, as the ISA understands that a destination address matching the internal LAN gets routed according to the “classless” route we defined on the ISA server.

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top