Why Do You Need to Create a Deny Rule to Support Some Custom Protocol Configurations?

You’re having problems connecting to a non-standard Web server. Most likely the problem is that the Web server you’re trying to connect to doesn’t work right with CERN proxies. While this isn’t a major problem, it’s common enough where it can cause an ISA or Forefront TMG firewall administrator some headaches if he’s not aware of the problem.

Now that you’re aware of the problem, what should you do? One solution is to configure a custom HTTP protocol that is unbound from the Web proxy filter. When the HTTP requests use this protocol, they are seen as purely NATed outbound requests to the destination Web server, and any problems related to CERN proxy compliance as bypassed.

You can create your own custom HTTP protocol by creating a custom protocol definition using TCP port 80 outbound and then not binding the Web proxy filter to the protocol.

However, this introduces a problem. If you have a built-in protocol that is bound to an Application Filter (such as the Web proxy filter) and a custom protocol that is not bound to a filter, both of which use the same destination port, you end up with what’s called overlapped protocols.

Overlapped protocols can cause a problem, because while the custom protocol will handle the initial connection from clients on the designated source network, any secondary connections from the client on the designated source network will be intercepted by the built-in protocol that is bound to the Web proxy filter.

This will cause the connections to fail in the same way that connections failed before you created the custom HTTP protocol definition that is unbound from the Web proxy filter. (this assumes that that you want to allow clients from the same source network to connect to servers using both custom and built-in protocols, if the rules were for different networks, the rule conflict would not exist because the rules using the overlapping protocols would not apply).

What’s the solution? Create a Deny rule after the allow rule that uses the custom protocol definition. The deny rule stops the processing for any secondary connections that are added to session established by the rule that used the custom non-filter bound protocol definition.

Pesach Shelnitz provides a good example of how this works in his blog post at:




Thomas W Shinder, M.D., MCSE
Sr. Consultant / Technical Writer

Prowess Consulting www.prowessconsulting.com

PROWESS CONSULTING | Microsoft Forefront Security Specialist
Email: [email protected]
MVP — Forefront Edge Security (ISA/TMG/IAG)

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top