I have seen some posts on the Microsoft TechNet Forums about messages that get stuck in the Draft folder or Insufficient system resources, and all of them are related to Exchange Server 2007 Back Pressure. In almost all cases Administrators believe that the guilty party for these issues is Exchange Server 2007 but it is not totally true because the Exchange Server controls system resources to prevent them from being overwhelmed and it allows the delivery process for current messages to be worked out. All these processes are part of the Back Pressure feature which is responsible for monitoring certain Exchange Server 2007 resources.
Which are the system resources monitored by the back pressure feature?
Free space on the hard disk drive where the message queue database resides.
Free space on the hard disk drive where the message queue database transaction logs reside.
Memory that is used either by the EdgeTransport.exe process or by all processes.
Number of uncommitted message queue database transactions that exist in memory.
Based on the monitored resources above, the back pressure feature uses 3 (three) different classification levels for resource utilization:
Normal
Action: The Exchange Server 2007 Server accepts new connections and messages.
Medium
Action: The resources sre starting to be overused. Back pressure is applied to the server in a limited manner. Messages from senders in the authoritative domain can flow, but the server will reject new connections and messages from other sources.
High
Action: A resource is being overused. All message flow will stop and the server will reject all new connections and messages.
Which roles are monitored by Back Pressure feature?
In Exchange Server 2007 there are 5 (five) roles: Edge Transport, Hub Transport, Mailbox Server, Client Access Server and Unified Messaging Server, but the Back Pressure feature monitors only two roles, that are: Edge Transport Server and Hub Transport Server.
The symptoms of the Back Pressure feature…
There are some symptoms to look out for when Back Pressure detects overused resources:
The messages sent by users are stuck in the Drafts folders, as shown in Figure 01.
Figure 01: All sent messages are stuck in the Drafts folders, one of the symptoms of the Back Pressure feature
This is the classic: when we try the connection to a Hub Transport or Edge Transport we can get the following message “4.3.1 Insufficient system resources”, as shown in Figure 02.
Figure 02: Insufficient system resources is the action of the Back Pressure feature
Another way to see Back Pressure in action is through the Event Viewer showing the resource level reached and explaining the values for Normal, Medium and High, and statistics as well. (Figure 03)
Figure 03: Back Pressure event in the Event Viewer of an affected server
Configuring Back Pressure…
Now, we know what the Back Pressure feature is let’s see how we can manage this feature. All the Back Pressure settings are found in a single file called EdgeTransport.exe.config. This file is located in the subdirectory bin of the Exchange Server 2007 installation folder. In Figure 04 we can see the parameters controlled by the Back Pressure feature.
Figure 04: Parameters related to the Back Pressure feature in the EdgeTransport.exe.config file
After any change in the configuration file we must restart the Microsoft Exchange Transport service to refresh the parameter settings of that file.
All the parameters shown in the Figure 04 above belong to 5 (five) settings groups, as follows:
General Options
Free Hard Disk drive space for Message Queue Database
Free Hard Disk drive space for Message Queue Database Transaction Logs
Memory that is used by the Edge Transport process
Memory that is used by all process –ok
Each group has some parameters that enable us to easily change the values of each monitored resource. The information about each parameter and value can be found in the Exchange Server 2007 help file. Note that the below information is based on Exchange Server 2007 RTM. Some of the default values below may be different in SP1. A brief explanation of each Parameter, default values and possible values are listed below:
General Options
Here we can enable or disable the Back Pressure feature and set up the resource monitoring interval.
Parameter | Default Value | Possible Values |
EnableResourceMonitoring | True (Enabled) | True (Enabled) or False (Disabled) |
ResourceMonitoringInterval | 00:00:02 (seconds) | At least 00:00:03 and at maximum 00:00:30 |
Table 1
Free Hard Disk drive space for the Message Queue Database
There is a default formula for this component:
100*(hard disk drive size – 4 GB) / hard disk drive size
This means we should have at least 4GB of free hard disk drive space always available for the message queue database.
Note:
The path of the Message Queue Database is stored in the QueueDatabasePath parameter in the same configuration file.
Parameter | Default Value | Possible Values |
PercentageDatabaseDiskSpaceUsedHighThreshold | 0 (indicates that the formula is in use). | 3 to 100 |
PercentageDatabaseDiskSpaceUsedMediumThreshold | 0 (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
PercentageDatabaseDiskSpaceUsedNormalThreshold | 0. (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
Table 2
Free Hard Disk drive space for the Message Queue Database transaction logs
In this parameter we control how much free space the Message Queue Database transactions logs utilize. By default the Back Pressure feature uses the following formula:
100*(hard disk drive size – 25*DatabaseCheckPointDepthMax) / hard disk drive size
Note:
The parameter DatabaseCheckPointDepthMax can be found in the same configuration file.
Parameter | Default value | Possible Values |
PercentageDatabaseLoggingDiskSpaceUsedHighThreshold | 0 (if zero, indicates that the default formula will be used) | 3 to 100 |
PercentageDatabaseLoggingDiskSpaceUsedMediumThreshold | 0 (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
PercentageDatabaseLoggingDiskSpaceUsedNormalThreshold | 0 (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
Table 3
Memory that is Used by the EdgeTransport.exe Process
The formula to calculate the memory used by the EdgeTransport.exe is the simplest Back Pressure feature. Actually, it is not a formula, but a rule: 75% of the total physical memory or 1 TB, the lower value wins.
Parameter | Default value | Possible Values |
PercentagePrivateBytesUsedHighThreshold | 0 (Indicates that the formula is used) | 3 to 100 |
PercentagePrivateBytesUsedMediumThreshold | 0 (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
PercentagePrivateBytesUsedNormalThreshold | 0 (if zero, indicates that the value is 2% less than the previous parameter) | 3 to 100. The value must be less than the previous parameter. |
Table 4
Memory that is Used by All Processes
We can control the memory that is used by All Processes; when memory utilization by all process reaches the value specified in PercentagePhysicalMemoryUsedLimit a process called message dehydration starts. By default, all message content is kept in memory. This process removes all the unnecessary elements (MIME Content) of those messages, and then memory used can be reduced.
Parameter | Default Value | Possible Values |
DehydrateMessagesUnderMemoryPressure | True (Enabled) | True (Enabled) or False (Disabled) |
PercentagePhysicalMemoryUsedLimit | 94 | 3 to 100 |
Table 5
Number of Uncommitted Message Queue Database Transactions That Exist in Memory
The “version buckets” are the message queue database transactions that are kept in memory. All changes that are made to the message queue database stay in memory until those changes can be committed to transaction log files.
Factors that can increase the version buckets may be virus issues, integrity of the message queue database, or hard drive performance.
Parameter | Default value | Possible Values |
VersionBucketsHighThreshold | 100 | 1 to 8000 |
VersionBucketsMediumThreshold | 60 | 1 to 8000. The value must be less than the previous parameter. |
VersionBucketsNormalThreshold | 40 | 1 to 8000. The value must be less than the previous parameter |
Table 6
How does Back Pressure work?
The Back Pressure feature uses 5 (five) stages to control the monitored resources. This means that the services will not be stopped at the first sign of trouble (Figure 05). For each stage certain actions that will be performed when Back pressure identifies a resource bottleneck. If there is no result at the current level and resource utilization is still increasing, the stage level will increase until the system resources come back to normal values. By default the levels are checked using the interval defined in the ResourceMonitoringInterval parameter of the EdgeTransport.exe.config file.
Figure 05: Stages that Back pressure performs before to stop momentarily some services
Below is a brief description of each stage. Remember that each stage checks their information against the parameters of the configuration file.
Stage 1
The memory utilized by the Edge Transport.exe process is validated. If higher than specified in the configuration file a process called Garbage collection will start. This process checks for unused objects that exist in memory, and removes those unused objects from memory.
Stage 2
The number of uncommitted message queue database transactions that exist in memory is validated. If higher than the configured value, an attempt is made to force the message queue database transactions that are in memory to be written in the transaction log files.
Stage 3
The utilization levels of all monitored resources are checked against the configuration file for normal levels of utilization. If overutilization persists; the resource with the highest level of utilization is acted upon. These actions are different in Edge Transport and Hub Transport, and you can see the action in the following tables:
Resource utilization level | Connections from other Hub Transport servers | Connections from other messaging servers | Store driver connections from Mailbox servers | Pickup directory and Replay directory submission | Internal mail flow |
Medium | Allowed | Rejected | Allowed | Rejected | Functional |
High | Rejected | Rejected | Rejected | Rejected | Not functional |
Table 7: Hub Transport
Resource utilization level | Connections from Hub Transport servers | Connections from other messaging servers | Pickup directory and Replay directory submission |
Medium | Rejected | Rejected | Rejected |
High | Rejected | Rejected | Rejected |
Table 8: Edge Transport
Stage 4
The memory utilization of the Exchange Transport process is validated against the config file. Even if we restart the Exchange Transport service the messages located in the Submission queue will not be processed automatically when the service starts. Other validation occurs in the message queue database transactions that are kept in memory. If higher than the normal level the following actions will occur: the transport dumpster will be disabled; message delivery to any remote destination that uses a remote delivery queue will be disabled.
Stage 5
If the memory utilization of the Exchange Transport process is still at a high or medium level or the memory utilization of all processes exceeds the value defined in the configuration file, the following actions will occur: DNS cache will be flushed from memory; and the message dehydration process occurs.
Conclusion
In this article we have seen that the Back Pressure feature helps the Exchange Server 2007 to prevent the inundation of system resources. When a monitored system resource exceeds the specified threshold, the Exchange Transport Services stop accepting new connections and messages, and concentrates on delivering the existing messages while the system comes back to normal levels.
