Managing Exchange 2003 With MOM 2005 (Part 2)
If you missed the other articles in this series please read:
In part one of this article, I gave you an introduction to the MOM 2005 state and diagram views and how they can assist you in monitoring and managing Exchange 2003. I also covered an overview of the reporting features since there are Exchange related reports that come as part of the Exchange management pack. In part two, I want to take a look at the important elements of the Exchange management pack, the component of MOM that contains all the information on how to correctly manage your Exchange 2003 infrastructure.
The Exchange Management Pack
The key thing to note about the Exchange management pack is that it has been developed by the Exchange Product Team with input from Microsoft Consulting Services (MCS) and Microsoft Product Support Services (PSS). Who better to create the Exchange management pack and all the knowledge it contains than Microsoft?
Contained within the Exchange management pack are rule groups, rules, views, knowledge, scripts, reports, and so on. In other words, all the components that you require to successfully manage Exchange are included for you. The Exchange management pack is designed to give you an out-of-the-box solution, although some tuning will probably be required in order to maximize performance. It is important to understand that configuring the Exchange management pack on one or more Exchange 2003 servers is made easy by using the Exchange Management Pack Configuration Wizard which I will cover in part three of this series. There are five primary rule groups contained within the Exchange 2003 management pack. Rule groups are simply sets of processing rules that are grouped together. The rules within the rule groups define how MOM collects, processes and responds to data. For Exchange 2003, the five primary rule groups for the Exchange management pack are:
- Availability and State Monitoring
- Exchange Event Monitoring
- Health Monitoring and Performance Thresholds
- Performance Counter Logging Rules
- Report Collection Rules
Availability and State Monitoring
The rules within this rule group are broken down into the following four key areas: MAPI logon check, mobility synthetic logons, mail flow and service monitoring.
MAPI logon checks are used to check the state of one or more of your mailbox stores. The checks occur every 5 minutes by default, allowing you to quickly know whether your mailbox stores are correctly servicing MAPI clients. The MOM 2005 agent will log onto each mailbox store in turn; the script that performs the MAPI logon process also records the result of the logon process via various event numbers. These event numbers are ultimately stored away in the MOM reporting database and can be used to produce Exchange availability reports. Table 1 below shows the important event numbers generated by the MAPI logon script and their meaning. Figure 1 below shows a small section of the Exchange Server Availability report and how the event numbers reflect as percentages in the various report columns.
Table 1: MAPI Logon Script Codes
Figure 1: Exchange Server Availability Report
Here you can see that 1918 successful MAPI logons occurred out of a possible 2016, which is a period of one week using 5 minute logon intervals. Therefore the availability of each mailbox store over this week period was 95.14%.
In addition to the MAPI logon checks, other synthetic logons are performed against the mobility services, namely Outlook Web Access (OWA), Outlook Mobile Access (OMA) and Exchange ActiveSync (EAS). With these checks, the MOM agent will log into a test mailbox and simulate an OWA, OMA or EAS connection. You can therefore quickly identify failures of these services and be ready to inform your users of any failures. You should note that the mobility synthetic logons require an Exchange 2003 front-end server running SP1 as well as a Secure Sockets Layer (SSL) implementation.
The localhost URL is monitored by default on an Exchange 2003 front-end server. For example, imagine we have a test mailbox called TESTSERVERMOM and wish to verify that OWA is working. The MOM agent on the Exchange front-end server will make a connection to https://localhost/exchange/testservermom as shown below in Figure 2. Note the use of HTTPS.
Figure 2: OWA Synthetic Logon Using Localhost
It is also possible, via the addition of registry keys on the front-end server, to instruct the MOM agent to monitor external URLs, such as https://email.domain.com. This is useful when monitoring access through a proxy and/or firewall. The required registry keys can be found in the Exchange Management Pack Guide for MOM 2005 for which I’ve provided a link at the end of this article.
It is important to understand the mail flow rules that are found within the Availability & State Monitoring rule group. MOM can be instructed to perform periodic mail flow tests between Exchange servers. The idea behind the mail flow tests is that every 15 minutes by default, mail flow test messages are sent from one Exchange server to another via MOM test mailboxes. Alerts are generated when the mail flow messages are not received when they are expected to be received and additionally the latency is recorded during the mail flow tests. The default latency threshold is 60 seconds, and a warning alert will be generated if this threshold is exceeded. The important thing to know about the latency is that it is only as accurate as the system clocks on the sending and receiving servers, so make sure that these are set correctly. Mail flow messages themselves are very small since they simply contain a subject line that reads ‘MailFlowMsg;’. Mail flow configuration is stored in the registry on each Exchange server and is created for you by the Exchange Management Pack Configuration Wizard.
Finally, there are also rules to check that the monitored Exchange services are functional which means that an alert can be raised if a particular Exchange service stops functioning. Like the mail flow configuration, the services to be monitored can be quickly configured by the Exchange Management Pack Configuration Wizard, which creates the following registry key on each Exchange server to reflect which services have been selected for monitoring:
This registry key contains a comma-separated list of services to be monitored as shown below in Figure 3. It’s therefore possible to manually alter which services are to be monitored by altering the contents of this registry setting, although you can also re-run the Exchange Management Pack Configuration Wizard to do the same thing.
Figure 3: Monitored Services
Exchange Event Monitoring
As you might expect, MOM can collect important information from the event logs of your Exchange servers and raise alerts accordingly. Exchange is capable of generating many thousands of event log entries but the Exchange management pack reduces this number down to monitor approximately 1,700 key event log entries that you need to know about. Figure 4 below shows the Exchange Event Monitoring rule group expanded out to reveal the many different categories of event log entries that are monitored by MOM.
Figure 4: Exchange Event Monitoring Rules
As you can see from Figure 4 above, there are 171 event rules for the Information Store service alone. These event rules will allow you to be alerted to the presence of any of these event log entries appearing on your Exchange servers, something that you’ll no doubt wish to happen fairly quickly if you have information store issues. The key to the event rules is the knowledge contained within them. When you receive an alert within the MOM Operator Console as a result of an event log entry being logged on your Exchange server, you can use this knowledge straight away for rapid issue resolution. You may have more junior members of your team that can use this knowledge to resolve issues first time, meaning less escalation within your support teams. What’s more, this knowledge comes directly from Microsoft, so you know that it’s correct; you’re no longer at the mercy of a Google search! Figure 5 below shows the knowledgebase information relating to the 9548 event log, which is seen when you disable a mailbox-enabled Windows account but do not set a master account SID. Notice how the information gives you causes and resolutions, including a link to a Microsoft KB article.
Figure 5: Exchange Management Pack Knowledge
Of course, there is also the ability to add custom company knowledge that remains with the particular event rule. For example, if a particular error occurs within your environment that requires a specific fix, this information can be added to the company knowledge tab which will then be associated with this error and can then be used to quickly resolve the issue should it happen again.
Health Monitoring and Performance
Within this rule group there are some of the common rules that you’d expect to find in any Exchange management product. For example, there are rules to check the free disk space on transaction log drives, SMTP queue drives, and so on. Once again, these checks are performed by a script which has a number of configurable parameters associated with it. By default there are parameters for specifying the percentage and number of megabytes of free space remaining before an alert is generated; warning alerts are generated if the free space drops below 20%, whilst error alerts are generated if the free space drops below 5%. Of course, these parameters are customizable to suit your environment. You’ll also find performance measuring rules present within this rule group to alert you if various Exchange queues exceed a predefined threshold. In this rule group you’ll find 15 rules such as:
- SMTP local retry queue > 50
- SMTP remote retry queue > 500
- MTA work queue > 50
- Categorizer queue > 50
- Public folder receive queue consistently > 10 deep
The Server Configuration and Security rule group is a sub-group of the Health Monitoring and Performance rule group and has rules that perform checks for problematic areas such as those listed below:
- Circular logging is enabled
- IIS Lockdown and URLScan have not been run
- SMTP Relaying is allowed
- Message tracking has not been enabled
- Transaction log files have not been backed up
- Mailboxes are homed on a front-end server
There are many more but you get the general idea. Now, those of you who currently run the Exchange Best Practices Analyzer (ExBPA) tool will be thinking that most of the above checks are also made by ExBPA and you’d be right of course. Now that ExBPA v2 has its own MOM management pack, it is expected that the above rules will be removed from future versions of the Exchange management pack. However, at this stage, it’s good to know that the Exchange management pack will perform such system checks should you not be using ExBPA for whatever reason.
Figure 6 below shows an example of a MOM alert in this rule group informing you that IIS Lockdown has not been run on a particular server.
Figure 6: IIS Lockdown Alert
Also within the Health Monitoring and Performance rule group are rules designed to raise alerts when Exchange server performance thresholds for common issues are exceeded. For example, there are rules such as:
- Disk write latency > 50ms
- Disk read latency > 50ms
- MSExchangeIS RPC latency > 200ms
- MSExchangeIS RPC requests > 25ms
- OMA response time > 60s
If you deploy the Exchange management pack, the above rules mean that you have knowledge found in whitepapers such as Troubleshooting Exchange Server 2003 Performance directly within MOM. Therefore, MOM will alert the administrator straight away when specific performance thresholds are met and importantly, the alert will also contain product knowledge as I’ve explained previously. This knowledge gives the administrator the information required to understand what the problem is and, more importantly, how to fix it.
Performance Counter Logging Rules
Performance counter logging naturally has a big role within the world of Exchange performance and as you’d expect MOM captures many other performance counters to aid you in understanding more about your Exchange environment. Figure 7 below shows the Performance Counter Logging Rules rule group folder expanded out to reveal the many different categories of performance counters that are collected by MOM. Rather than just discuss all the different types of categories, what I’d like to do is to focus on one particular category and show you how MOM can help you understand the performance of your Exchange environment.
Figure 7: Performance Counter Rules
Take the Client Monitoring sub-rule group as an example. Within this rule group there are 22 rules associated with collecting Outlook 2003 RPC performance data. By default, Outlook 2003 sends RPC information back to the Exchange 2003 server. This information includes how many RPCs succeeded, how many failed, the RPC latency, and so on. MOM gathers this RPC information via the 22 processing rules which allows you to generate reports that detail the Outlook 2003 client experience. The MOM report titled Exchange 2003 Outlook Client Monitoring shows how this data can be used to report on the total number of attempted RPCs, the total of failed RPCs, the percentage of successful RPCs, the average server RPC latency and the percentage of latency for different time values. Part of an example report is shown below in Figure 8.
Figure 8: Exchange 2003 Outlook Client Monitoring Report
So it’s not difficult to see how the performance data gathered by MOM can be used to report back on the real Outlook 2003 client experience. Obviously MOM has many other processing rules for the more obvious performance related issues, such as generating an alert if the CPU usage threshold on your Exchange server has exceeded a certain limit over a time period you specify.
Report Collection Rules
Finally we have report collection rules which obtain configuration information about your Exchange environment. For example, there are report collection rules to determine the number of storage groups and mailbox stores on each server, as well as documenting on which disks those mailbox stores physically reside. How many times have you needed to find out how large your mailbox stores are, or how large your users’ mailboxes are? Now you can find this information very quickly, all from the MOM Operator Console as well as by producing reports.
The report collection rules are also responsible for gathering message tracking log statistics, public folder statistics and other important information such as operating system information as it relates to Exchange management and performance. For example, is the /3GB switch set correctly in boot.ini?
Figure 9 below shows you an example of how report collection rule information is presented within the MOM Operator Console. In this case, we’re looking at the storage group and mailbox store configuration information. It’s very easy in the MOM Operator Console to pull up such information quickly and therefore areas like system documentation become much easier too.
Figure 9: Report Collection Rule Results
In part two of this three part series, I’ve taken a look inside the Exchange management pack to show you what it can do for you in the world of Exchange management. The Exchange management pack is one of the largest management packs available for MOM 2005 and it’s not difficult to see why; the management pack is literally packed with information, features and knowledge that you can really make use of when monitoring your Exchange infrastructure. In the last part of this series, I will be taking a look at the Exchange Management Pack Configuration Wizard, the ExBPA management pack and Active Directory client-side monitoring.
Exchange Management Pack Guide for MOM 2005:
Troubleshooting Exchange Server 2003 Performance:
If you missed the other articles in this series please read: