Protecting Microsoft Exchange with ISA Server 2004 Firewalls



Protecting Microsoft Exchange with ISA Server 2004 Firewalls:
Integrating the ISA Firewall into an Established Network Infrastructure


By Thomas W Shinder M.D.


Nobody likes to start from scratch. This is especially true if you have a well established network and firewall infrastructure that’s working for you. Why would you want to go and change everything just to add a new application layer intelligent firewall to your setup? Things are working already and you haven’t been successfully attacked for at least 6 weeks.


This is something I come across a lot when recommending ISA firewalls to organizations that already have a firewall and network infrastructure in place that’s pretty much working for them. When they do come around to realizing that they’ll benefit from the added protection of an application layer intelligent firewall like ISA, they choose to install it in what I call “crippled” mode, where the ISA firewall acts only as a reverse proxy server to protect HTTP/HTTPS (SSL/TLS) communications. This configuration is like buying a Corvette and taking three tires off of it because it “goes too fast”.


The reason for this “crippling” is usually because they think its easier to place the ISA firewall on their network using this method. In this “reverse proxy mode”, the ISA firewall does not need to be in the inbound or outbound path, and they don’t need to worry about “zero-day” exploits against Windows operating systems. These are some valid concerns, primarily related to ISA Server 2000 installations. Fortunately, the concerns they had with ISA Server 2000 melt away with ISA Server 2004 firewalls.


The problem with ISA Server 2000 firewalls was that they had a simple view of the networks to which they were attached. A network was either trusted (and included in the Local Address Table or LAT) or it was untrusted (not in the LAT). All communications between LAT hosts were not inspected by the ISA Server 2000 firewall. In addition, all communications between LAT and non-LAT hosts were NAT’d. You could not route requests between LAT and non-LAT hosts. Even if you set up a DMZ segment, where you could route, simple stateful packet filters and not the full range of application layer firewall stateful inspection that the ISA Server 2000 firewall provided, controlled the communications.


The great news is that ISA Server 2004 firewalls are completely different than ISA Server 2000 firewalls. Firewall rules are applied to all interfaces and there is no more LAT. There are no “trusted” networks; all networks are untrusted and all communications moving through the ISA Server 2004 firewall are exposed to firewall policy. Creating DMZs is a simple affair now. Unlike ISA Server 2000, you can now easily place your front-end Exchange Servers on a DMZ segment and apply firewall policy to secure communications between the front-end and back-end Exchange Servers.


Speaking of Microsoft Exchange, it’s clear that ISA Server 2004 should be considered the firewall for protecting Microsoft Exchange Servers. ISA Server 2004 includes a number of technologies aimed at specifically protecting Microsoft Exchange Servers. These include:



  • Forms-based authentication
  • Delegation of basic authentication
  • SSL to SSL bridging (SSL termination)
  • Advanced HTTP Security Filtering
  • URL protection
  • OWA/OMA/ActiveSync wizards that create secure publishing rule by default
  • Secure Exchange RPC filtering
  • And lots more!

A key feature in ISA Server 2004 Exchange deployment scenarios is that it provides sophisticated application layer protection for Microsoft networking services and services regardless of where you place it on the network. You can benefit from ISA Server’s ability to protect against:



  • HTTP application layer attacks: Code Red, Nimda, HTR overflows, directory traversal attacks, buffer overflow attacks, chunked transfer encoding attacks, cross-site scripting attacks, malicious URLs, malicious HTTP content, high-bit encoding attacks, WebDAV attacks.
  • SMTP application layer attacks: spam flood attacks, malicious attachment attacks, SMTP buffer overflow attacks, SMTP disk flood DoS attack, spammer open relay attack, brute force closed relay attack, all worms and worm variants
  • DNS application layer attacks: DNS zone transfer from untrusted sources, DNS buffer overflow attacks, DNS malformed query attack, DNS malformed packet attacks
  • POP3 application layer attacks: POP3 buffer overflow attacks, POP3 malformed command attacks
  • RPC application layer attacks: Microsoft BLAST, Microsoft Blaster variants, Nachi, RPC worm variants and morphs.

In addition, in each scenario, the ISA Server 2004 firewall enables you to:



  • Block access to all Windows executables
  • Block access to Web sites based on keywords on Web pages and URLs
  • Block access to Web sites based on signatures in Request and Response headers and data
  • Enforce encrypted channel for remote access secure Exchange RPC Outlook clients
  • Block FTP downloads or enable FTP downloads for specified users
  • Allow only the HTTP methods (PUT, GET, etc) you want to allow; block all others for everyone, or on a user/group basis – granular control is yours
  • Enforce strict RPC compliance to halt RPC worms as soon as they arrive at the ISA 2004 firewall; this also has the added benefit of stopping DCOM from moving through the ISA 2004 firewall into and out of the corporate network

Only when the ISA Server is configured in a single-NIC configuration is the full array of sophisticated application layer protections not available. In the limited single-NIC configuration scenario, the ISA Server protects only against HTTP related attacks.


In this article, we will discuss a number of network topologies that demonstrate placement options available to you. These topologies include:



  • High-speed packet filtering firewall at the Internet edge with a back-end ISA Server 2004 firewall. The back-end ISA Server 2004 firewall protects the front-end and back-end Exchange Servers, both of which are located on an Internal network segment
  • High-speed packet filtering firewall at the Internet edge, with a front-end Exchange Server on a perimeter network segment between the front-end firewall and the ISA Server 2004 firewall on the back-end. The back-end Exchange Server is located on the protected Internal network segment behind the ISA Server 2004 firewall machine
  • High-speed packet filtering firewall at the Internet edge, with an ISA Server 2004 firewall on the back end. The multihomed back-end ISA Server 2004 firewall includes an External interface, an Internal interface and a perimeter network interface. The front-end Exchange Server is located on the perimeter network segment directly connected to the ISA Server 2004 firewall and the back-end Exchange Server is located on the Internal network segment
  • High-speed packet filtering firewall at the Internet edge, and a non-ISA general-purpose firewall working on the back end. An ISA Server 2004 firewall is configured to work in Web Proxy mode to provide reverse Web proxy services that enables access to OWA, OMA and ActiveSync sites located on Exchange Servers on Internal networks located behind the non-ISA firewall on the back-end
  • High-speed packet filtering firewall on the front-end. ISA Server 2004 firewalls located in a back-to-back configuration behind the high-speed packet filter. Firewall and Web Proxy chaining are configured to provide a level of security not typically available with non-ISA firewalls. The front-end and back-end Exchange Servers are both located on the Internal network behind the back-end ISA firewall
  • High-speed packet filtering firewall on the front-end and a non-ISA firewall on the back end. ISA Server 2004 application intelligent firewalls protect dedicated services segments containing front-end and back-end Exchange Servers

A key feature in each of these scenarios is that the current firewall and routing infrastructure is left in place. Reconfiguration of firewalls and routers is kept at a minimum. The ISA Server 2004 firewall placement relatively transparent. This transparency is critical because the high-speed packet filter based firewalls at the Internet edge must be kept in their current locations. High speed packet filters are able to meet throughput requirements for multigigabit connections to the Internet.


The front-end high-speed packet filters can quickly “pass packets” to perimeter or backbone networks. The high-speed packet filters pass packets to multiple devices located behind them after performing rudimentary network layer stateful filtering. Packet load is distributed among a greater number of back-end devices. This configuration where high speed packet filters handle high volume traffic and multiple back-end devices distribute the high volume into multiple smaller volumes allows the back end devices to provide the higher level of security required to protect servers and services located behind the secondary firewalls. The back-end devices do not need to meet the same performance requirements as the front-end devices because they do not handle the same level of traffic.


Let’s take a closer look at each of these ISA Server 2004 firewall topologies.


High Speed Packet Filter Firewall at Internet Edge/ISA Server 2004 Firewall as Back-end Firewall Protecting Microsoft Exchange


The first scenario has the high-speed packet filter based firewall on the front-end and a sophisticated application layer filtering ISA Server 2004 firewall on the back-end. The ISA Server 2004 firewall provides the advanced application layer protection required for the Microsoft Exchange Server.


This network topology is straightforward. The front-end high-speed packet filters move data inbound and outbound through the corporate network very quickly. Between the high-speed packet filters is a perimeter network segment separating the front-end firewalls from the back end firewalls. The figure below shows a back-end ISA Server 2004 firewall in front of a network services segment containing front-end and back-end Exchange Servers.


Only a minimum amount of configuration is required on the front-end firewalls. The front-end firewalls should be configured to:



  • Forward all inbound requests for the Exchange OWA sites to the external address of the ISA Server 2004 firewall
  • Forward all inbound SMTP messages for the Exchange organization to the external address of the ISA Server 2004 firewall
  • Forward all inbound requests for IMAP4 services to the external IP address of the ISA Server 2004 firewall
  • Forward all inbound requests for POP3 services to the ISA Server 2004 firewall
  • The ISA Server 2004 firewall is configured is configured to use an upstream router that forwards Internet bound requests to the front-end packet filters
  • An optional configuration, which confers a high level of security, is for the front-end high-speed packet filters to forward all mail related (HTTP/HTTPS/SMTP/IMAP3/POP3/RPC) connections to the ISA Server 2004 firewall. While the advanced application layer filtering ISA Server 2004 firewall cannot provide the same throughput as high speed packet filters, the ISA Server 2004 firewall provides a superior level of security.
  • In each of these configurations, the back-end ISA Server 2004 firewall publishes the front-end Exchange Server

As you can see from the figure below, the ISA Server 2004 firewall can be easily placed on the edge of the services segment or an Internal network segment with only minimal configuration on the front-end high-speed packet filtering firewalls.


Figure 1: Front-end High Speed Packet Filter Firewalls/Back-end ISA Server 2004 firewalls



 


Front-end High Speed Packet Filter/Back-end ISA Server 2004 Firewall and Front-end Exchange Server in the perimeter network Segment


This scenario describes a network topology where the front-end packet filtering firewalls remain on the Internet edge, while the ISA Server 2004 machine acts as a back-end firewall protecting the back-end Exchange Server. The front-end Exchange Server is located on a perimeter network segment between the front-end packet filter and the back-end ISA Server 2004 application intelligent firewall. In this scenario, you must take extra care to harden the front-end Exchange server because the only simple stateful packet filters protect it from Internet attackers..


This configuration requires very little reconfiguration on the front-end high-speed packet filters. Changes to be made include:



  • The front-end packet filter forwards inbound connections for the OWA Web site to the IP address used by the front-end Exchange Server in the perimeter network segment
  • The front-end packet filter forwards inbound connections for Exchange SMTP services to the front-end Exchange Server in the perimeter network segment
  • The front-end packet filter forwards inbound POP3 connections to the front-end Exchange Server in the perimeter network segment
  • The front-end packet filter forwards inbound IMAP4 connections to the front-end Exchange Server in the perimeter network segment
  • The front-end Exchange server is a member of the same domain as the back-end Exchange Server. The back-end ISA Server 2004 firewall is configured to allow intradomain communications through the firewall, as well as allowing the SMTP, POP3, IMAP4 and HTTP/HTTPS communications to the back-end Exchange Server. Security can be added between front-end and back-end Exchange Server by using IPSec for front-end/back-end communications.

This is a popular configuration. The front-end Exchange Server provides a unified namespace and a single point of entry for back-end Exchange Servers. A measure of security is realized because the front-end Exchange Server does not contain user mailboxes.


However, placing the front-end Exchange server on the perimeter network segment introduces an enhanced security risk because:



  • The high speed packet filter on the Internet edge cannot provide the high level of application layer security required to protect modern networking services, such as those hosted on the front-end Exchange Server
  • The front-end Exchange Server is a member of the Internal network domain that contains user accounts and Active Directory. Extending the security zone represented by the Active Directory into a lower security perimeter network segment is generally considered poor security practice

Figure 2: Front-end Packet Filter/Back-end ISA Server 2004 firewall and Front-end Exchange in perimeter network



 


Front-end High Speed Packet Filter/Back-end Multihomed ISA Server 2004 Firewall with Front-end Exchange on Trihomed Perimeter Network Segment


In this scenario, the front-end high speed packet filter remains on the Internet edge and the ISA Server 2004 application intelligent firewall remains on the back end. The difference between this scenario and the last one is the front-end Exchange Server is placed on a trihomed perimeter network segment connected to the back-end ISA Server 2004 firewall. This network design provides a higher level of firewall protection for the front-end Exchange Server. In contrast to the rudimentary packet filters on the front-end high-speed packet filtering device, the ISA Server 2004 firewall’s sophisticated application layer filtering and stateful inspection protects the Exchange Server.


The perimeter network segment on which the front-end Exchange Server is located can have either a route or NAT relationship with the perimeter network between the front-end high-speed packet filters and the back-end ISA Server 2004 firewall. The advantage of the NAT relationship is that it hides the IP address used by the front-end Exchange Server.


Changes on the front-end high-speed packet filter include:



  • If a route relationship is used between the trihomed perimeter network segment and the perimeter network segment between the front-end and back-end firewalls, then the high speed packet filters on the front end can be configured to forward SMTP, POP3, IMAP4 and OWA, OMA and RPC over HTTP connections to the actual IP address of the front-end Exchange Server on the trihomed perimeter network segment
  • If a NAT relationship is used between the trihomed perimeter network segment and the perimeter network segment between the front-end and back-end firewall, then the high-speed packet filters on the front end can be configured to forward SMTP, POP3, IMAP4, OWA, OMA and RPC over HTTP connections to the IP address on the external interface of the ISA Server 2004 firewall.
  • The ISA Server 2004 firewall is configured with a default gateway on its external interface that forwards Internet bound packets to a router that forwards them through the high speed packet filters

This configuration works well with ISA Server 2004 firewalls because:



  • Firewall Policy is applied to all interfaces on the ISA Server 2004 firewall.
  • You can place a domain member computer on the trihomed perimeter network segment and allow the intradomain communications between the perimeter network and Internal network segment without allowing access to all protocols.
  • You can configure either a route or NAT relationship between the perimeter network and Internal network segment. In a route relationship, you configure Access Rules allowing the front-end server to communicate with the back-end; in a NAT relationship, you configure Web and Server Publishing rules. In general, Access Rules provide a higher level of flexibility and route relationships provide a higher level of protocol support; not all protocols/applications work properly through NAT devices

The downside of this design is that the Internal network security zone containing the user database and Active Directory is extended into a relatively lower network security zone. However, this design still represents an improvement over the previous scenario because the ISA Server 2004 firewall protects the front-end Exchange Server.


Figure 3: Back-end ISA Server 2004 firewall in Trihomed perimeter network Configuration



 


Front-end High Speed Packet Filtering Firewall/Back-end Non-ISA Firewall and ISA Server 2004 Firewall in Web Cache Configuration in the Perimeter Network


In this scenario, the front-end high-speed packet filters remain on the Internet edge. The back-end firewalls are non-ISA general purpose firewalls. Between the front-end high-speed packet filters and the back-end non-ISA firewalls is an ISA Server 2004 firewall in a unihomed cache configuration. The figure below shows the firewall placements for this configuration.


This configuration provides an organization with an existing financial investment in front-end and back-end firewalls the advanced application layer inspection and unique protection for Microsoft Exchange Services provided by the ISA Server 2004 firewall. The unihomed (single NIC) ISA Server 2004 firewall is configured in “cache mode”. This configuration disables the majority of the strong application layer protection provided by the ISA Server 2004 firewall, but leaves the HTTP application intelligence intact.


The unihomed ISA Server 2004 firewall in cache configuration can act as a reverse (and forward, if you wish) Web proxy and provides support for remote access connections to Exchange OWA, OMA, ActiveSync and RPC over HTTP services. You will not be able to reverse proxy incoming SMTP, POP3 or IMAP4 connections because publishing these protocols requires the ISA Server 2004 firewall to be configured as a firewall.


Although the bulk of firewall functionality is removed from the ISA Server 2004 firewall when it is configured in a unihomed cache configuration, incoming and outgoing Web connections continue to be exposed to the deep HTTP inspection provided by the HTTP security filter, and inbound SSL to SSL bridging can be configured to provide stateful application layer inspection for SSL connections. SSL to SSL bridging prevents hackers from hiding exploits in SSL tunnels.


Configuration of the front-end high-speed packet filters includes:



  • Forwarding inbound HTTP and HTTPS communications to the IP address of the unihomed ISA Server 2004 firewall in cache configuration
  • Not forwarding SMTP, POP3 or IMAP4 connections to the unihomed ISA Server 2004 firewall in cache configuration

The unihomed ISA Server 2004 firewall in cache configuration should be configured to:



  • Publish the front-end Exchange Server. If there is a NAT relationship between the perimeter network and the front-end Exchange Server, then forward the connection to the external address of the back-end non-ISA firewall. If there is a route relationship between the perimeter network and the front-end Exchange Server, then forward the connection to the actual IP address of the front-end Exchange Server
  • The ISA Server 2004 firewall does not need to be configured with a default gateway if the front-end high speed packet filtering firewalls NAT between the Internet and the perimeter network. If the high speed packet filters route between the Internet and the perimeter network, then the unihomed ISA Server 2004 firewall’s default gateway should be configured with a gateway that routes Internet bound requests back to the high speed packet filters

The back-end non-ISA firewall should be configured to:



  • Route the inbound connection from the ISA Server 2004 firewall to the front-end Exchange Server if there is a route relationship between the perimeter network and Internal network
  • If there is a NAT relationship between the perimeter network and the Internal network, then configure the non-ISA firewall to publish or perform a reverse NAT to forward the connection to the front-end Exchange Server
  • The back-end non-ISA firewall does not need to be configured to use the unihomed ISA Server 2004 firewall as its default gateway. This allows you to leave the current default gateway configuration on the back-end firewall as is, without changes.

This configuration is ideal for organizations with a heavy investment in front-end and back-end firewalls who want the benefits of a powerful ISA Server 2004 application intelligent firewall.. In addition, the front-end and back-end Exchange Server at both on the Internal network, so the internal network security zone isn’t extended into the perimeter network. This provides much better security then when the Internal network security zone is extended into a lower security zone.


Figure 4: Unihomed ISA Server 2004 firewall in Cache Configuration in perimeter network



 


Front-end High Speed Packet Filters with Back to Back ISA Server 2004 Firewalls on the Back End


In this scenario, the high-speed packet filters on front end remain at the Internet edge. ISA Server 2004 firewalls are placed in series behind the front-end packet filters to create a back-to-back ISA Server 2004 firewall configuration. This setup allows the front-end packet filters to pass packets quickly, while providing a highly secure perimeter network segment between the ISA Server 2004 firewalls and Internal network segment behind the back-end ISA Server 2004 firewall.


This design is useful for those organizations with an existing front-end firewall infrastructure, but no strong back-end firewall infrastructure providing intelligent application layer protection to a public access services segment that communicates with networking services on a secure Internal network. A back-to-back ISA Server 2004 firewall configuration located behind the current firewall infrastructure can provide this level of security.


Configuration on the front-end packet filters includes:



  • Forwarding all communications for OWA HTTP and HTTPS services to the external address of the front-end ISA Server 2004 firewall
  • Forwarding all communications for Exchange hosted SMTP, POP3 and IMAP4 services to the external interface of the front-end ISA Server 2004 firewall

The front-end ISA Server 2004 firewall is configured to:



  • Publish the OWA site on the front-end Exchange Server using a Web Publishing Rule
  • Publish the SMTP, POP3 and IMAP4 services on the front-end Exchange Server using Server Publishing Rules
  • Configure the front-end ISA Server 2004 firewall’s default gateway to use a router that routes Internet bound requests back to the front-end high speed packet filtering firewalls

The back-end ISA Server 2004 is configured to:



  • Publish the OWA site on the back-end Exchange Server using a Web Publishing Rule
  • Publish the SMTP, POP3 and IMAP4 services on the back-end Exchange Server using Server Publishing Rules
  • Configure the back-end ISA Server 2004 firewall’s default gateway as the internal IP address of the front-end ISA Server 2004 firewall

This configuration provides enhanced protection for the front end Exchange Server located on the perimeter network segment located between the two ISA Server 2004 firewalls. In addition, the Internal network segment benefits from an even higher level of protection because it is protected by two application layer firewalls.


Even if the front-end ISA Server 2004 firewall were to fail, the back end ISA Server 2004 firewall continues to protect the internal network. The drawback of this configuration is the Internal network security zone is extended into the perimeter network segment, which represents a lower security zone than the Internal network security zone.


Figure 5: Back to Back ISA Server 2004 Firewalls Behind Fast Packet Filters



 


Front-end High Speed Packet Filters on the Front-end, non-ISA Firewalls on the Back End and ISA Server 2004 Firewalls Protecting Department LANs and Services Segments


The scenario incorporates a three or more tiered firewall topology. At the front end, high-speed packet filtering firewalls perform initial packet filter based screening. Located behind the high-speed packet filter firewalls are the non-ISA general purpose firewalls. The network between the front-end high-speed packet filters and the back-end non-ISA firewalls is a low security corporate backbone or perimeter network segment. Low security hosts, such as honeypots or IDS systems might be placed on this segment.


A second security zone is located behind the back-end non-ISA firewalls and the ISA Server 2004 firewalls. This represents a public access or anonymous access security zone where public access servers can be place. This segment may also represent a secondary corporate backbone.


ISA Server 2004 firewalls are placed on the edge of departmental LANs and network services segments and provide strong inbound and outbound user/group based access control and superior application layer security for Microsoft Exchange and other Microsoft network services. This is a highly secure configuration because it takes advantage of a well worn military concept that the level of defense should increase as you get closer to the highest security core services.


Because it is protected only by high-speed packet filter based firewalls, the outermost network perimeter has the weakest level of firewall security. The second perimeter is protected by a non-ISA general-purpose firewall, which provides a higher level of security than the fast packet filters. These secondary firewalls might provide a rudimentary level of application layer filtering. The ISA Server 2004 firewalls are placed near the innermost security zones, where the highest level of protection is required. This layered approach insures that the highest level of protection where it is needed most: at the perimeter of the network client and services segments.


Configuration of the fast packet filters and second-tier firewalls varies from network to network. However, this scenario is consistent all other scenarios covered in this paper: it is quite easy to drop the ISA Server 2004 firewall behind existing non-ISA firewalls. This allows ISA Server 2004 firewalls to be placed deep in the network behind the secondary backbone or perimeter network and in front of the high value targets deep within the corporate network.


Figure 6: Multi-tiered Firewall Configuration with ISA Server 2004 Firewalls Providing Application Layer Protection



 


Conclusion


ISA Server 2004 is an advanced application layer firewall that includes a number of features that make it a compelling option for protecting Microsoft Exchange Servers. This article described several scenarios demonstrating that an ISA Server 2004 firewall can be placed virtually any where on the corporate network with minimal disruption to the current firewall and network topology. A common theme in each of the six firewall topologies discussed is that the ISA Server 2004 firewall works well with existing firewall setups. Only a handful of rules need to be changed on the fast packet filters. Even in infrastructures that have invested heavily in a front-end and back-end firewall configuration, an ISA Server 2004 firewall can be easily added to provide the needed application layer security demanded on networks subjected to todays sophisticated hacker attacks.

Leave a Comment

Your email address will not be published.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top