Preventing P2P and Instant Messaging programs from hijacking your network with ISA 2004 Firewalls

Preventing P2P and Instant Messaging programs
from hijacking your network with ISA 2004 Firewalls

by Greg Mulholland

Instant Messaging has become one of the most useful tools of communication in these times. The pure speed in which information can be transferred from one end of the globe to the other is phenomenal. However, we are becoming aware of some major implications of the availability of this technology.

Network and Firewall Administrators have been facing a battle to uphold the integrity and productivity of their networks. Some of the major issues they have found with these potentially dangerous applications (P2P, IM’s) are the potential to disclose corporate information (source code etc) in a non mediated forum, the misuse of company resources, legal issues, possible virus incursion and simply the fact that it is another (flavor of the month) type point of attack, potentially jeopardizing the entire network.

This article will describe how in simple terms we can leverage a new feature of ISA Server 2004 to prevent these types of applications clogging our internet pipe and exposing our company/network to the above issues.

To begin the process we need to be sure of a few things. Firstly you should be aware that in ISA Server 2004 there is already a pre-defined protocol called “MSN Messenger” which allows traffic on port 1863 for Windows Messenger and MSN Messenger to connect to the Internet. When creating a firewall policy we need to specify which protocols we will allow our users access to. It follows that the MSN Messenger protocol should not be configured in our primary rule.

By restricting the accessible protocols from our internal network we make it more secure. How you ask! Kazaa and its variants will connect to many IP addresses on various port numbers in the a variety of ranges. If we restrict outbound access to only the predefined ports our users need access to i.e. SMTP, POP3, HTTP and others, these programs will not be able to connect in the normal manner.

Don’t believe me? Try an “open access rule” and watch the firewall logs as Kazaa connects, then try a “restricted access rule” and compare the difference.

For the simplicity of the article I will illustrate only with Windows Messenger but the same concepts will still apply to Kazaa, MSN Messenger, Yahoo and others.

So we have setup the restricted access rule to allow only the “specified” protocols and we see that Windows Messenger will not connect. Hooray!

What happens though when the smarter users try to get around this by setting the Windows Messenger client to use the Proxy server to connect? This is very simple to do in most applications and has been working a treat up till now. All they need to do is to enter the firewalls/proxy’s IP address or name and the proxy port, normally 8080. They also have the option to enter their proxy authentication details in if required.

Once these details are entered and the user tries to connect to the service, guess what, there in, Doh!

Thankfully, Network and Firewall Administrators are now able to prevent these applications from running on their network and being a potential risk to the enterprise. ISA 2004 firewall’s ability to block specific signatures essentially gives us the power to block applications that tunnel traffic over HTTP with the use of specific patterns in the request headers, request body, response headers and response body. To do this we need to go back to the restricted access rule we have defined and right click and then select “Configure HTTP”.

Here we are able to configure the HTTP Policy for this rule (which includes all internal users).


To exempt users from such rules, we could simply create two internal firewall rules and apply the condition to a specific group i.e. temps, admins, programmers etc).

In specifying signatures to block, I found this handy link which gives some of the more prominent ones.

So following that guide we create a policy to block the Windows Messenger signature and of course apply the changes to the firewall.

You can create as many as you want to depending on your needs. I have only used two for testing purposes to date. Remembering that Windows Messenger was able to connect using the proxy settings, its time to go back and test our signature blocking.

As you can see we have had some success. Windows Messenger was not able to connect even though it was using the proxy server settings we defined earlier.

What does it all mean? Well the good news is gone are the days when our networks are subject to misuse from our very own. With the new ISA 2004 firewall and the HTTP policy options we are able to keep this type of unwanted activity off our network.

While this is an effective way of achieving this, some people might like to have more control and or reporting features. In this case there are a number of software add-ons that work extremely well with ISA Server to achieve this goal.

Rouge Aware by Akonix is a P2P and IM sniffer that can detect the use of such programs, even after you thought you had locked your network down. Its brother (or sister) product is L7 Enterprise which is the tool that allows you to create and enforce IM and P2P network policies. Two extremely handy products. For the sake of time that is all I will say about these products, however Tom and Deb Shinder have put together some information on both which you can find below, along with links to download trial versions for your own testing.

Hope it all works well for you!

Greg Mulholland

If you would like us to email you when Greg Mulholland releases another article on, subscribe to our ‘Real-Time Article Update’ by clicking here. Please note that we do NOT sell or rent the email addresses belonging to our subscribers; we respect your privacy.

Leave a Comment

Your email address will not be published.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top