Content Caching with Forefront Threat Management Gateway (TMG) 2010
Forefront Threat Management Gateway (TMG) 2010, when deployed as a web proxy server, can be configured to cache frequently requested web objects in memory and on disk in order to improve web browsing performance and to reduce bandwidth utilization. Web content caching is available for both forward and reverse proxy scenarios. Content caching brings with it different benefits in each of these deployment scenarios.
How It Works
With content caching enabled, when the TMG firewall receives a web proxy request the firewall will first attempt to fulfill the request from the cache. If the requested content does not exist in the cache, it will make the request of the origin server as normal. When the web server responds, TMG will return the content to the client, and then store any cacheable content in the cache. Subsequent requests for the same content will be served directly from the cache and returned to the client at LAN speeds, eliminating the need to make a trip to the origin server to retrieve the content. This improves page loading speeds for end users and reduces bandwidth consumption on Internet links.
Enabling Content Caching
Content caching can be enabled and configured using the Web Access PolicyWizard, which can be run once the Getting Started Wizard has completed.
If you elected not to use the Getting Started Wizard to configure TMG, open the TMG management console and highlight Web Access Policy, then click Configure Web Access Policy.
When prompted by the Web Access Policy Wizard, select the option to Enable the default Web caching rule, and then click Cache Drives….
The only requirement for enabling content caching is having at least one NTFS formatted volume on which to store the cache file. Highlight the volume you wish to store the cache file on, and then specify the maximum size of the cache in megabytes (MB). It is generally recommended that you allocate a minimum of 100MB in addition to .5MB for each Web Proxy client using the system.
The content cache can be up to 64GB per volume. You can specify multiple cache files on additional volumes, if required. For optimal cache performance, best practices dictate that you store the cache file on a volume that does not contain the system partition.
Caching is implemented using Cache Rules, which function much the same as firewall policy rules. You can view, edit, and add rules by clicking the Web Caching: link in the Web Access Settings window.
Select the Cache Rules tab and you will see an ordered list of rules that ends with a Default rule. Each cache rule is evaluated in order from top to bottom, with the first rule that matches the request being processed. This is an important point to remember to prevent unexpected cache rule processing.
The Web Access Policy Wizard automatically creates a cache rule that works well in most cases. To view the properties of this rule, highlight the rule and choose Edit…. On the General tab you will find that the rule is enabled, and on the To tab the destination is the External network, indicating that this rule will apply to any request made to the External network.
The Cache Store and Retrieval tab defines explicitly if and how to cache the requested content. You can control what content is retrieved from the cache and what to do with requests for objects that are not currently cached, and you can choose the option to cache dynamic content, content for offline browsing, and content requiring user authentication.
The HTTP tab is used to enable and configure HTTP caching. Here we can specify how long objects will live in the cache.
The FTP tab is used to enable and configure FTP object caching. Here we can specify how long objects downloaded via FTP will live in the cache.
The Advanced tab is used to configure a limit on the size of objects in cache. This will prevent large downloaded files from consuming excess amounts of cache. In addition there is an option to Cache SSL responses. This applies only to bridged SSL traffic from web publishing rules. The TMG firewall does not cache SSL responses in forward (outbound) proxy scenarios.
On rare occasion you may encounter a web site that does not work well with a caching proxy server. To prevent content from a particular web site from being cached, create a new cache rule that applies only to the specific destination in question.
Select the option Never, no content will ever be cached.
Make certain to place this cache exception rule first in the ordered list of cache rules. This will prevent another cache rule from matching before the exception rule, causing the content to be cached unexpectedly.
Content Caching and Enterprise Arrays
To take full advantage of content caching for enterprise arrays, TMG includes support for the Cache Array Routing Protocol (CARP). CARP essentially allows for multiple, discrete cache volumes on each array member to be addressed as a single, logical cache drive. This makes the cache more efficient and eliminates cache content duplication. CARP is implemented in one of two ways – client side or server side.
- Server-side CARP – When an array member receives a web proxy request for URL, it will run the URL through an algorithm that will return an index value to indicate which array member the cached content is most likely stored. The array member will then submit a web proxy request for the content to that array member. If that array member does not have that content in its cache, it will retrieve the content from the origin server, return it to the requesting array member, and then store the content in its configured cache drive. Future requests for the same URL will subsequently be served from cache.
- Client-side CARP – A more efficient way of implement CARP is to have the client pre-determine which array member the cached content is most likely stored on. In order to take advantage of this, web proxy clients must be configured to use auto detection (WPAD), or be configured to retrieve the proxy autoconfiguration script from TMG. This script contains the algorithm required for the client to compute the index value itself, which allows it to forward the request to the appropriate array member directly. This eliminates the need for array members to forward requests to other array members, reducing CPU and network utilization for the array.
To determine cache effectiveness, use the Windows performance monitor to observe the Cache Hit Ratio (%) counter of the Forefront TMG Web Proxy object. This will indicate the percentage of requests that are retrieved from cache. To assess the recent performance of the cache, use the Cache Hit Ratio for last 10K Requests (%). In addition, TMG includes the highly detailed Forefront TMG Cache object, which includes many counters with which to gather additional detail about the performance of the content cache.
The native TMG reports also include high-level information about cache utilization and performance. When creating a TMG report, include the Traffic and Utilization category to view this information.
Content caching is a simple and effective way to improve performance for end-users and reduce bandwidth utilization on Internet links. Enabling caching is made easy with the Web Access Policy wizard and requires only an NTFS formatted volume as a prerequisite. For web sites that don’t play well with caching proxies, cache rules can be configured to exempt specific destinations from caching altogether, if required. For enterprise arrays, CARP can be enabled to allow individual cache drives on each array member to be addressed as a single, logical cache. Monitoring the efficiency and performance of the cache is simple using the Windows Performance Monitor and the native TMG reporting capabilities.