Let’s continue with our discussion about how to troubleshoot ISA firewall performance issues.
After confirming that nothing else could be responsible for the performance issues, you can move to the ISA firewall machine itself and see if it’s the ISA firewall device or software or software configuration is responsible for your Internet performance woes. Remember a key trick to solving ISA firewall performance and connectivity problems:
In the vast majority of cases the connectivity or performance problem is not an ISA firewall issue, but something else entirely
Once you appreciate the wisdom and truth of this statement, you’ll end up solving your problems much faster.
Now you need to go to the ISA firewall device and open the Performance Monitor and add ISA Server related counters. I typically start with the ISA performance console that comes pre-built with the ISA firewall software and add to that. You’ll need to know what counters are important, which is hard to do when you’re not sure what the problem is. The best place to start is by looking at the default counters and comparing those values with tolerances found in the counters list at http://www.microsoft.com/technet/prodtechnol/isa/2004/plan/isa_2004_perftroubleshooting.mspx If the default counters aren’t useful then you’ll have to start coming up with some hypothesis about what the problem might be.
Check the log files on the ISA firewall. This includes the Alerts tab in the ISA firewall console and the Event Viewer. Many times you’ll find helpful clues to what the problem is in one of these areas. If you see an error, do a search on that error and use the built in tools in the Event Viewer to find out if there is a KB article on this problem. You’ll be pleasantly surprised how often you’ll find a solution just by doing this. If you do find something in the Alerts tab or in the Event Viewer, but can’t find any information after doing a Search for that error, then post a note on the ISAserver.org Web boards or mailing list. Many times one of us has seen the same thing but Microsoft doesn’t have a KB article up yet on the subject.
One of the most useful tools in your firewall toolkit is Network Monitor (or Ethereal if you prefer). Many times a network trace is the only way to solve a problem, such as when user sessions seem to time out randomly. In this example, the problem is most often the Web server sending the ISA firewall a disconnect message because of problem with the Web server software itself. Network monitor traces can tell you if there are long delays in responses, which might cause state table timeout problems. NetMon is also very useful in determining where there seems to be an excessive amount of a single type of traffic, such as pings, DNS, or SMTP. Of course, you need to spend some time baselining what is normal for your environment because you can tell what is excessive, but many times you can identify unusual traffic without any formal baseline assessments.
If NetMon and the default PerfMon counters do not provide you useful information, then start adding counters based on what your hypotheses are about what the problem might be. Remember, there’s always an answer to the problem, the only issue is that you don’t know it yet. But you will.
When there are performance issues related to the ISA firewall, these are the things you might want to check into first:
- Confirm that the DNS settings on the ISA firewall are correct
- Confirm that you’re not under some sort of DNS attack, or that some server on your network is sending excessive DNS queries through the ISA firewall. The most common reason for this is that SMTP servers are attempting to resolve fake domain names to which they need to send NDRs
- Confirm that the ISA firewall is not under some sort of denial of service attack. You should be able to easily recognize a DoS from your network traces and the ISA firewall’s log files and Alerts because a DoS will generate Connection Limits Alerts
- Confirm that you have the latest NIC drivers for your ISA firewall NICs
- Confirm that you have hard-coded the speed settings on the ISA firewall’s NICs
- Confirm that the UPS the powers the switches to which each of the ISA firewall’s NICs is in good health. Often when the power goes out on the switch, there are errors in communications after that which can be solved by restarting the ISA firewall (I don’t know the technical reason why this is true, but I’ve seen the problem many times)
- Confirm that Path MTU Discovery is enabled on the ISA firewall
- Confirm that third-party software or complex rule sets aren’t pegging the processor
- Create allow rules, do not create or minimize the number of Deny rules (least privilege)
HTH,
Tom
Thomas W Shinder, M.D.
Site: www.isaserver.org
Blog: http://blogs.isaserver.org/shinder/
Book: http://tinyurl.com/3xqb7
MVP — ISA Firewalls