Introducing the ISA Server 2000 Application Layer Filtering Kit

Introducing the ISA Server 2000 Application Layer Filtering Kit

By Thomas W Shinder M.D.

 

Download the entire set of ISA Server 2000 Application Layer Filtering Kit documents at:

http://www.isaserver.org/img/upl/alfkit/alfkit.zip (34 MB)

 

Read the ISA Server 2000 Application Layer Filtering Kit documents online at:

http://www.isaserver.org/img/upl/alfkit/

Chapter 1:
Better Together: Protecting Microsoft Networks with Microsoft ISA Server 2000 Application Filtering Firewalls

Chapter 2:
Block Unwanted Email and Viruses with the SMTP Filter and Message Screener

Chapter 3:
Prevent Attacks Against Microsoft Exchange Servers using ISA Server 2000 RPC Filters

Chapter 4:
Prevent Virus and Hacker Attacks against Secure Web and OWA sites with SSL Bridging

Chapter 5:
Block Hacker Attacks Against Web and OWA Sites with URLScan 2.5

Chapter 6:
Block Buffer Overflow Attacks Against Published DNS and Mail Servers with DNS and POP3 Application Layer Filters

Chapter 7:
Warn and Protect Against Hacker Attacks Using ISA Server 2000 Intrusion Detection

Chapter 8:
Prevent Attackers from Learning About Network Infrastructure Names using the Link Translator

Chapter 9:
Increasing Security by Extending ISA Server 2000 Application Layer Filtering

Remember the good old days (if they ever existed) where you could put up a packet filtering firewall that performed simple stateful filtering and feel like you were secure? Attackers back then weren’t nearly as good as they were now, and there weren’t so many of them. In the past you could harden the Internet facing servers on the internal network and publish them using simple packet filters without worrying too much about sophisticated application layer attacks aimed against the published server.

Traditional packet filtering firewalls just can’t do the job of providing the level of protection your network requires. While many organizations have high-speed packet filtering firewalls in place to pass packets as fast as possible to and from the internal network and the Internet, the simple fact is that these organizations have traded security for performance. A comprehensive network defense in depth posture has the high-speed stateful filtering firewall on the Internet edge, the intelligent application filtering and inspection firewall (like ISA Server 2000 firewalls) on the LAN edges, and host-based hardening on the Internet facing servers that have been published by the intelligent application layer filtering ISA Server 2000 firewall.

Unwanted email and application layer based network attacks are the two biggest threats to corporate networks today. Criminal spammers steal bandwidth, processing resources, administrator time and mail server disk space by sending unsolicited commercial email to the corporate network. Unwanted email also can contain dangerous payload that includes viruses, worms and trojans. Unwanted email can lead to lost employee productivity and possibly expose your organization to attorneys.

Application layer attacks attempt to leverage weaknesses in the server service the attacker is trying to whack. A traditional stateful filtering firewall isn’t able to determine the validity of the application layer communications. As long as a packet filter is configured to allow the inbound connection attempt, and the session state appears valid (pseudo-session in the case of UDP), then the traditional stateful packet filtering firewall lets the exploits through to the server on the internal network.

How to allow remote users to access services on the internal corporate network and also prevent users from downloading dangerous content and uploading proprietary corporate information? How do you get comprehensive protection at the network perimeter so that exploits never make their way into the corporate network?

Get the Book!

Intelligent Application Layer Filtering and Inspection Firewalls Prevent Attacks Against Network Services

The answer to comprehensive security at the perimeter is the application layer inspection or application layer filtering firewall (sometimes referred to as stateful inspection). The stateful inspection or application layer filtering firewall is able to block attacks by looking for anomalies in the application layer header and data sections of a communication. Application layer filtering firewalls build on the features of the traditional stateful filtering firewalls and enforce both valid connection states and valid application layer communications.

Application layer filtering firewalls are required to protect networks from modern attackers because attackers now focus their efforts on developing exploits against weaknesses in the services they attack. Attackers use a variety of application layer specific methods to exploit known and unknown weaknesses in server services to disable servers or take control of them. An application layer filtering firewall is able to examine the application layer commands and data and determine whether the content or commands being sent to a server on the corporate network fall outside the bounds of valid connection attempts.

Imagine that a company wants to allow their off-site employees access to the full range of Exchange Server features using the full Outlook MAPI client. You can’t create packet filters to secure the connection because the nature of the RPC communications requires a large number of static packet filters to be created, which could put the organization at risk. The only secure way this can be accomplished with a stateful filtering-only firewall is to require remote users to establish a VPN connection to the corporate network and connect to the Exchange Server through the VPN link. VPN connections can potentially provide significant obstacles because of their inherent complexity and end-user confusion on how to install, maintain and manage a VPN client connection.

An application layer firewall that understands how the Outlook client communicates with the Exchange Server using the RPC protocol and Exchange UUIDs can manage the Outlook/Exchange communications and do it in a secure fashion. The application layer filtering firewall that understands the nature of valid RPC communications is able to drop exploits directed against the Exchange Server’s RPC interfaces and allow only valid Outlook connections.

Application layer filtering can also be used to prevent inappropriate communications from leaving the corporate network. Users may try to send proprietary corporate data out of the corporate network to individuals that should not have this information, or users may try to download files located on the Internet that contain viruses, worms or trojans that can be used to attack internal network servers. An intelligent application layer filtering firewall can prevent losses due to uploading sensitive corporate information or downloading dangerous code.

Application layer filtering firewalls can be used instead of, or in conjunction with traditional stateful packet filtering firewalls. When used instead of a traditional stateful packet filtering firewall, the application layer filtering firewall can stop application layer exploits at the Internet edge and prevent attacks at the front line, so that attack code never reaches perimeter networks or internal corporate networks. On the other hand, you may wish to leave high performance (but low security) network layer stateful packet filtering firewalls on the Internet edge and put the intelligent application layer filtering firewalls on the edge of corporate network segments that require the highest level of application layer filtering protection against Internet-based attackers.

How ISA Server 2000 Application Layer Filtering and Deep Content Inspection Protects Microsoft Networks

ISA Server 2000 represents the model of an application layer filtering firewall. Because ISA Server 2000 is a software based firewall, it is able to quickly accommodate the processing and inspection overhead that comes with deep application layer inspection and filtering. ISA Server 2000 application layer filtering firewalls have the ability to block a number of application layer attacks and unwanted email right out of the box. In addition, you can expand the already high level of application layer security provided by ISA Server 2000 firewalls by installing security add-ons.

An ISA Server 2000 firewall is also ideally suited to protect Internet facing Microsoft services. These include Internet Information Server services, Exchange Server services, SharePoint Portal Server services, VPN server services and many more. ISA Server 2000 firewalls leverages the unique level of understanding Microsoft has of its own network services and uses this knowledge to provide an impressive level of protection for Microsoft networks and network services.

This ISA Server 2000 Application Layer Filtering Kit focuses on ISA Server 2000’s sophisticated application layer filtering and inspection mechanisms and how they protect Microsoft servers and services. There are a number of ways ISA Server 2000 firewalls protect corporate networks against today’s application layer focused attacks. These include:

  • Blocking unwanted email and SMTP buffer overflows with the SMTP filter and SMTP Message Screener
  • Securing remote full Outlook MAPI client connections with the secure Exchange RPC filter
  • Preventing attacks from hiding inside SSL tunnels using ISA Server 2000 SSL to SSL bridging
  • Blocking suspicious Web connections at the perimeter using URLScan for ISA Server 2000
  • Protecting against DNS and POP3 buffer overflow attacks using the ISA Server 2000 DNS and POP3 filters
  • Improving incident response with ISA Server 2000 intrusion detection and filtering
  • Preventing attackers from learning about your internal computer network names and using that information against you by using the ISA Server 2000 Link Translator
  • Expanding ISA Server 2000 firewall application layer filtering and inspection with high security add-ons
  • Block Buffer Overflow Attacks and Unwanted Email with the SMTP Filter and SMTP Message Screener

    Buffer overflow attacks against server services is one of the most common methods attackers use to disable a network service and potentially take control of the server running the network service. An attacker can craft a packet containing oversized SMTP commands and send these to an SMTP mail server. If the mail server implementation has a known or unknown buffer overflow weakness, the attack could disable or take over the server.

    ISA Server 2000 comes with the SMTP filter that contains a pre-built list of SMTP commands and insures that no inbound SMTP connections are made that exceed the legitimate size of a valid SMTP command. The SMTP filter blocks the buffer overflow attempt at the firewall and prevents the attack from getting past the ISA Server 2000 firewall.

    Unwanted email represents one of the major threats to network security and stability today. Unwanted email clogs email servers and impairs overall employee productivity. ISA Server 2000 includes the SMTP Message Screener that can with alone, or together with another unwanted email filtering solution to provide an anti-unwanted email defense in depth solution. The SMTP Message filter blocks unwanted email based on source email account or email domain, keywords in the subject line or body, and attachment type, name or size.

    Read the ISA Server 2000 Application Layer Filtering Kit document

    Chapter 2: Block Unwanted Email and Viruses with the SMTP Filter and Message Screener for more information on how the SMTP filter and SMTP Message Screener application layer filters protect the corporate network.

    Securing Full Outlook MAPI Client Connections with the Secure Exchange RPC Filter

    Corporate network employees enjoy using the same email client application regardless of their location. The full Outlook MAPI client allows users on the internal network full access to the entire range of Exchange Server features when connected to the corporate network. Users often become dissatisfied and experience decreased productivity when they leave the office and must use another email client application to access information stored on the Exchange Server.

    Traditional stateful packet filtering firewalls cannot be configured to allow remote users the high level of productivity afforded by the full Outlook MAPI client because of the large number of ports that must be allowed inbound and outbound. The traditional packet filtering firewall does not understand the Outlook/Exchange RPC connections and has no way to secure these connections. Either organizations must allow the Outlook MAPI clients VPN access to the network, or risk being infected by RPC worms and other exploits designed to take advantage of the large number of open ports on the stateful packet filtering firewall.

    In contrast to the traditional stateful packet filtering firewall, ISA Server 2000 is an intelligent application layer filtering and inspection firewall that understands Outlook/Exchange RPC communications. The ISA Server 2000 secure Exchange RPC filter allows valid inbound connections from the full Outlook MAPI client to the Exchange Server and blocks illegitimate connection attempts. Because ISA Server 2000 is a sophisticated application layer filtering firewall, it can allow remote users full access to the array of Exchange Server services using the full Outlook MAPI client and protect against RPC worms and other RPC related exploits.

    Read the ISA Server 2000 Application Layer Filtering Kit document

    Chapter 3: Prevent Attacks Against Microsoft Exchange Servers using ISA Server 2000 RPC Filters for more information on how the secure Exchange RPC filter perform intelligent application layer filtering and inspection to enable secure remote access for remote Outlook MAPI clients.

    Prevent Attacks from Hiding inside SSL Tunnels using ISA Server 2000 SSL to SSL Bridging

    Conventional stateful packet filtering firewalls can be configured to allow incoming connections to secure Web servers on the corporate network. Secure Web servers require that the connection between the Web client on the Internet and the Web server on the internal network use SSL to encrypt the username, password and data moving between the two. The encrypted information is protected because intruders cannot read the information moving inside the SSL tunnel.

    The problem is that the conventional stateful packet filtering firewall cannot evaluate information inside the SSL tunnel. Even third party application layer firewalls are unable to inspect the contents of the communications between the Web client on the Internet and the Web server on the internal network because the application layer firewall is unable to determine the contents of the SSL tunneled communications.

    ISA Server 2000 provides a unique level of protection and application layer filtering and inspection for secure Web servers. The ISA Server 2000 SSL to SSL bridging feature allows the application layer filtering and inspection features of ISA Server 2000 to decrypt the SSL tunnel and inspect the connection to insure that only valid communications are passed through the firewall. When the communications pass inspection, then the ISA Server 2000 firewall re-encrypts the communications and forwards them. Attackers are no longer able to hide attack code inside an encrypted SSL tunnel.

    Read the ISA Server 2000 Application Layer Filtering Kit document

    Chapter 4: Prevent Virus and Hacker Attacks against Secure Web and OWA sites with SSL Bridging for more information on how ISA Server 2000 firewalls use SSL to SSL bridging to protect secure Web servers on the corporate network.

    Block Suspicious Web Connections at the Perimeter with ISA Server 2000 URLScan

    Application layer attacks are the most common type of attack directed against Web servers. Attacks can create special requests aimed at exploiting known and unknown weaknesses in Web server software. An application layer filtering firewall should be able to review the HTTP header information and data and be able to determine when a potential attack is taking place.

    ISA Server 2000 firewalls use a special version of URLScan to review HTTP requests that are forwarded to Web servers on the internal network. This version of URLScan works very much like the URLScan that is installed on Internet Information Services Web servers. The advantage of using URLScan on the ISA Server 2000 firewall is that Web based attacks are stopped at the perimeter and are never forwarded to the Web server on the internal network. Only communications that pass the URLScan filtering mechanism are allowed through the firewall and forwarded to the corporate Web server on the internal network.

    Read the ISA Server 2000 Application Layer Filtering Kit document

    Chapter 5:
    Block Hacker Attacks Against Web and OWA Sites with URLScan 2.5
    for more information about how to use URLScan on the ISA Server 2000 firewall to block HTTP exploits at the network edge.

    Protect Against DNS and POP3 Buffer Overflow Attacks with the DNS and POP3 Filters

    Buffer overflow attacks are the most popular attacks launched against corporate servers exposed to the Internet. Buffer overflows can disable server services or even allow attackers to take control of the server. An application layer filtering firewall should be able to detect buffer overflow attacks at the perimeter and stop them before they ever reach the server on the corporate network.

    ISA Server 2000 firewalls include the DNS and POP3 application layer filters. These filters protect DNS and POP3 servers from buffer overflow attacks launched against them from Internet intruders. The DNS and POP3 application layer filters can automatically protect your corporate DNS and POP3 servers when you publish them to the Internet.

    Read ISA Server 2000 Application Layer Filtering Kit document

    Chapter 6:
    Block Buffer Overflow Attacks Against Published DNS and Mail Servers with DNS and POP3 Application Layer Filters
    to learn about how the DNS and POP3 application layer filters are used to protect your network.

    Improve Incident Response with ISA Server 2000 Intrusion Detection and Alerting

    In addition to network layer attacks, Internet-based attacks can use network layer attack methods to compromise your firewall and gain access to the corporate network. ISA Server 2000 includes a number of intrusion detection filters that allow you to detect these network layer attacks and prevent them from disabling the firewall or accessible internal network resources.

    ISA Server 2000 can also be configured to Alert the security administrator of an ongoing attack. ISA Server 2000 Alerts can send information to the Event Logs and send an email to a security administrator who can take quick, corrective action. In addition, ISA Server 2000 Alerts can be configured to automatically run a script or program that can mitigate the effects of the attack.

    Read ISA Server 2000 Application Layer Filtering Kit document

    Chapter 7:
    Warn and Protect Against Hacker Attacks Using ISA Server 2000 Intrusion Detection
    to learn about how the ISA Server 2000 firewall detects and warns of ongoing network layer attacks and how to configure Alert actions to mitigate the negative effects of the attack.

    Hide Internal Network Names using the Link Translator

    Network attackers use a variety of methods to gain access to information on your corporate network. Different attackers have different motivations; some want to destroy information or disable the network, either “just for fun” or for political or economic reasons (business competitors). Others do not wish to do damage, but instead want to steal information. This is often done for profit (corporate espionage). The latter care more about stealing private information than destroying or disabling network services.

    Some Web sites return the names of private internal network servers. Attackers can use this information to help them steal private and proprietary information from your network. The ISA Server 2000 Link Translator can correct this problem by leveraging ISA Server 2000’s application layer filtering firewall features to return only public names to external users accessing the Web sites.

    Read ISA Server 2000 Application Layer Filtering Kit document

    Chapter 8:
    Prevent Attackers from Learning About Network Infrastructure Names using the Link Translator
    to learn how the ISA Server 2000 Link Translator can be used to hide internal network names from Internet attackers.

    Expanding ISA Server 2000’s Deep Inspection Capabilities with Security Add-ons

    Network attackers continue to develop new and more sophisticated methods to attack corporate networks. Modern application layer firewalls must be able to adapt to evolving application layer attacks. ISA Server 2000 firewalls are ideally suited to meet this challenge. ISA Server 2000 allows you to expand the level of application layer protection provided by the firewall. This expandability enables the ISA Server 2000 firewall to meet the application layer firewall filtering needs of today and keep pace with changes taking place in the network attacks landscape.

    Read ISA Server 2000 Application Layer Filtering Kit document

    Chapter 9:
    Increasing Security by Extending ISA Server 2000 Application Layer Filtering
    to learn more about how powerful security add-ons can be used to allow you to stay one step ahead of Internet attackers.

    Get the New Book!

    Summary

    ISA Server 2000 is a sophisticated, intelligent application layer filtering firewall that can help protect networks against the network attacks of today and tomorrow. ISA Server 2000 firewalls can be used instead of traditional stateful filtering firewalls or in conjunction with an existing packet filtering firewall infrastructure. ISA Server 2000’s application layer filtering and inspection mechanisms provide the ideal level of network security and protection for Internet facing Microsoft servers and services, and provide powerful protection as part of a unwanted email and network attack defense in depth strategy.

    I hope you enjoyed this article and found something in it that you can apply to your own network. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=6;t=002218 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! –Tom

    If you would like us to email you when Tom Shinder releases another article on ISAserver.org, subscribe to our ‘Real-Time Article Update’

    by clicking here. Please note that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy!

    About The Author

    Leave a Comment

    Your email address will not be published. Required fields are marked *

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

    Scroll to Top