If you missed part 1 of this article series please read Optimizing ISA performance (Part One) – Nine Basic Steps.
In this part 2 of the two part series on ISA performance, we will cover performance tweaks that improve ISA Server 2004, these guidelines specifically take into account the services that ISA server serves. Because of ISA’s multi layered architectural approach and sophisticated policy engine, it is extremely important that resources are carefully controlled and that only unnecessary resource consumers are disabled. It is strongly advised that, before making changes to your ISA server, a backup is made and the restore is tested.
ISA 2004 provides clients and networks with Security, Caching and filtering services. As vendors start to exploit the potential of the ISA 2004 exposed APIs, new advanced applications for ISA 2004 are found.
Each resource has a capacity limit, and as long as all resources are consumed below their limit, the best possible system performance is achieved. When one of these limits is exceeded a decrease in performance occurs. This can be resolved by increasing the available capacity to the resource which is lacking in capacity. Once this is done, performance should take a positive turn once again.
For ISA to perform at its optimal capacity, ISA needs to be continuously monitored and run within a comfortable range. The steps to monitoring ISA were covered in the two part article about hardening ISA 2004 (Hardening ISA Server 2004 (Part 1) & Hardening ISA Server 2004 (Part 2)). Once ISA monitoring is working, you are well on your way to obtaining an optimal ISA Server.
To ensure that ISA is running within a comfortable range, the ISA server capacity should be well planned. This involves considering the available and actual bandwidths on every network that is linked to an ISA Server computer, the number of users and various application metrics; with the availability of bandwidth on networks being the most important aspect. The number of users is less indicative of the needed capacity as they are not all following the same usage patterns simultaneously. Initially, planning for maximum network capacity may be conservative, because capacity requirements often increase over time. To accommodate future growth, you should also plan for processing power upgrades. For this reason it is often recommended to plan for 1.5 times the amount of users, at very least, that your ISA server environment will cater for.
Recently a network with ISA 2004 as its gateway was built for over 30,000 users. To ensure that the servers would handle the load, the specification was carefully considered. Microsoft has released sizing tools for most products including ISA 2004 and ISA 2006 and these tools can be used to size your ISA installation. This tool is used as a guideline and it must be noted that there are still other tweaks discussed in this document that will improve ISA’s performance.
It is recommended that a thorough assessment of the ISA server performance be done using the sizing tools supplied by Microsoft. After the assessment, the respective hardware that will satisfy the requirements resulting from the assessment should be procured. The ISA server should then be installed and after, tested using performance measuring tools. These tools can be used to establish baselines that can be reordered and then later used to compare reported data. Before establishing these baselines it is pertinent that all third party software that is not linked to ISA or the respective windows installation is removed. Third party software that is installed on ISA can have counter effects on performance, especially if the installation is untested. A staged approach is recommended as the next step in moving some of the more tolerant users onto the system. These pilot users will help identify glitches and performance issues if any arise.
In terms of hardware, my tips for before procuring the hardware are to ensure that the server’s disks, RAM and CPU are optimally specified to work with each other. If budget allows, I would advise to use 15000 RPM disks for an environment with a high caching requirement. Note that CARP (Cache Array Routing Protocol) is used to enhance the availability and performance of a scaled ISA solution, this in turn improves performance. The faster the FSB (Front side bus) the quicker ISA can handle the transfer of data and the processing of data.
The three factors of performance, namely RAM, CPU, and Disk, have limits for which they perform to and when these limits are reached the performance is degraded. Using monitoring applications like MOM and/or performance monitor with thresholds that alert the security professional, will ensure that the upgrade or tweak can be applied in a timely fashion.
Packet Filtering
Using transport layer stateful filtering instead of Web Proxy filtering reduces ISA’s CPU utilization for the same traffic patterns by a factor of 10. Together, stateful filtering and application filtering can be used in parallel to provide granular control over performance.
Bandwidth and traffic
If the enterprise’s bandwidth totals up to more than 25 mbps (T1) then a dual CPU of Xeon 2.4 Ghz is recommended, in many cases the bottleneck is not the CPU but actually the bandwidth.
Caching
If ISA cache is not used, turn it off. Note caching uses drive I/O. ISA also uses RAM when caching and for this reason, when tweaking ISA, both the RAM and Disk should be monitored.
Application and Web Filters
ISA 2004 Server makes use of application filters when inspecting security at the application level. Typically this is a dynamic-link library (DLL) that registers with a specific protocol port. Scanning happens when traffic is passed through the port. Application filters pass traffic though the usermode OS stack.
Because application filters are ISA firewall processing extenders, they will have an impact on the ISA server’s performance. If possible use an ISA firewall rule instead of a filter.
When no application filter is assigned to a port, the traffic undergoes standard TCP stateful filtering of the TCP/IP header information. To increase the performance of your ISA server the application level scanning can be turned off. Because of the feature richness of the application filters that ISA offers, a small overhead is experienced. If your hardware is under spaced than it is recommended that the hardware resources be increased to a more adequate level or that the application filtering be turned off.
Third party application and web filters can be a reason for performance degradation so it is recommended that, before either option is installed, the solution be tested thoroughly before implanting live.
Logging
Note SQL logging will use up more CPU cycles, especially for instances that have verbose logging enabled. Monitor the CPU usage and ensure that your CPU is correctly balanced for the load. It must also be noted that logging will consume bandwidth. Consistently hosting live reports reflecting the logged information will also consume resources and for this reason it is recommended that an alternate hosting server and network card is used for this sole purpose especially in large environments.
Internet link bandwidth
1 Mbps
5 T1 (7.5 Mbps)
25 Mbps
T3 (45 Mbps)
SQL transactions per second
25
188
625
1,125
SQL transaction bandwidth
92 kilobits per second (Kbps)
700 Kbps
2.3 Mbps
4.2 Mbps
Table 1.1: The table above is a Microsoft extract that depicts suggested usage for bandwidth consideration
Networks
When using networks, always try to use the highest available network link available. If there is a 1 GB link to the switch than use it, especially if high LAN access is required for intranets, internal browsing, caching and LAN access.
Tuning
- The /3GB Boot.ini Switch should be used for large systems with over 2 GB of memory and Windows Server 2003 (see article Q171793).
- In terms of mass authentication, RADIUS has the lowest taxing authentication system than Kerberos and NTLMv2. These are strong systems and are recommended over older, less secure systems.
- Use SSL where it’s needed only. This is a great PKI solution and must be used where needed, remembering that the cycle is taxing on the system.
- Use OWA over RPC if possible as OWA is 100kb per connection and RPC is 500kb per connection.
- Reduce the usage of SSL bridging if it is not needed. Double CPU cycles can be reduced.
- Use load balancing to reduce the load on one ISA system.
Microsoft has made an effort in helping customers when sizing and improving the performance of their ISA server. A significant amount of information is available on the internet specifically addressing performance improvements for ISA server and for the supported operating systems. If you are planning on installing ISA for the first time, an appliance may be the right solution for your organization as these pieces of equipment have been specifically designed to remove the guesswork from the capacity planning and performance measurement and management of ISA server.
For more detailed information refer to the link that follows: http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/bestpractices.mspx
Summary
In this second article of ISA 2004 performance tweaking, we covered hardware tweaks and performance enhancements. We also looked at tips that can be used when optimizing an ISA installation along with other tips from the field that will assist you when improving your installation of ISA server.
If you missed part 1 of this article series please read Optimizing ISA performance (Part One) – Nine Basic Steps.
