Exchange 2010 Client Throttling Policies

Introduction

Exchange 2007 introduced a feature called RPC Client Throttling to allow administrators to manage end-user performance by preventing client applications, such as Outlook for example, from sending too many Remote Procedure Call [RPC] requests per second to Exchange, causing the server to suffer in terms of performance. When Exchange determines that a client is having a negative effect on the server, it will send a “back-off” request to the client telling it to delay sending any additional requests for a specified time (maximum of 2000 milliseconds) in order to reduce the performance effect on the server.

In Exchange 2010, Client Throttling has been much improved, monitoring and controlling much more than just RPC requests. Its purpose is still to ensure that users are not intentionally or unintentionally straining Exchange and that users share resources proportionally.

There is also Message Throttling in Exchange that restricts the number of messages and the number of connections that can be processed by an Exchange Transport server. In this article we will be talking only about Client Throttling.

What is Monitored?

So what does Exchange 2010 now monitor that Exchange 2007 didn’t? As I mentioned before, with Exchange 2010 it’s not just RPC requests that are monitored, but 9 different components:

• Anonymous access
• Cross-Premises Access (CPA)
• Exchange ActiveSync (EAS)
• Exchange Web Services (EWS)
• IMAP
• POP
• Outlook Web App (OWA)
• RPC Client Access (RCA)
• PowerShell

Let’s start with the first 8 which have similar policy parameters:

Table 1: Exchange Client Throttling Parameters and Components

Note:
Although there are dozens of configurable parameters, in this article we will address most of them, but not all – only the main ones.

Each cell of this table is a combination of <Component Acronym> like EAS, POP or OWA and <Throttling Parameter> such as MaxConcurrency. The end result is OWAMaxConcurrency or EASPercentTimeInAD for example.

If you are wondering if Unified Messaging [UM] users are throttled or not, the answer is “yes, they are”. UM users are considered Exchange Web Services users and, as such, are throttled by EWS parameters.

MaxConcurrency

The MaxConcurrency policy parameter specifies how many concurrent connections a user can establish to an Exchange server at one time.

A connection is first established when Exchange receives a request from a user and is closed when a response is sent back. If the user tries to make more concurrent connections than the ones set in his/her policy, the new connections fail but existing ones are not affected.

The valid values for <Component Acronym>MaxConcurrency range from 0 to 2147483647. To set it to unlimited (unthrottled), as with all the other parameters, simply set the value to $null.

For example, EASMaxConcurrency is set to 10 by default, meaning users can “only” have up to 10 mobile devices connected at the same time using ActiveSync.

PercentTimeIn*

The PercentTimeInCAS, PercentTimeInAD, and PercentTimeInMailboxRPC parameters are not as straightforward as the MaxConcurrency one. These indicate the percentage of a minute that a user can spend respectively:

• Running Client Access Server code;
• Running LDAP requests against Active Directory [AD];
• Running mailbox RPC requests.

Let’s take two examples: EWSPercentTimeInAD (50 by default) and OWAPercentTimeInCAS (150 by default).

The value of 50 indicates that for every period of 1 minute, the Web Services process can spend 30 seconds with LDAP requests.

But what about that value of 150? Does that mean that per each minute a process can spend 90 seconds consuming resources? Not exactly as we have to consider concurrent requests! A value of 150 allows two or more processes to spend 90 seconds in total per each minute. If a user had 3 processes running at the same time and running tasks that take the same time, this means each process could spend 30 seconds per each minute running CAS code (3×30 = 90 seconds or 150% of 60 seconds).

To complicate things a little more, the value of PercentTimeInCAS is always higher or the same as PercentTimeInAD and PercentTimeInMailboxRPC. This is because PercentTimeInCAS overlaps PercentTimeInAD and PercentTimeInMailboxRPC.

If you think about it, for a process to make an AD or RPC call, it has to be running CAS code already. This means that when a process does an LDAP request, for example, it is both consuming PercentTimeInAD and PercentTimeInCAS. For this reason, the value of PercentTimeInCAS should always be set higher than both PercentTimeInAD and PercentTimeInMailboxRPC.

PowerShell

Now, that we looked at the first eight components there is only one left: PowerShell. Here, we have 4 main parameters:

• PowerShellMaxConcurrency – this parameter depends on the context of which the user is running PowerShell. If it is a Remote PowerShell session, then it controls how many remote shell sessions a user can have opened at the same time. If the user is using Exchange Web Services, it controls how many cmdlets a user can have running at the same time.
• PowerShellMaxCmdlets – defines how many cmdlets a user can run per time period (set by PowerShellMaxCmdletsTimePeriod);
• ExchangeMaxCmdlets – although not under the PowerShell “section”, this parameter is similar to PowerShellMaxCmdlets but only for Exchange cmdlets and, therefore, should be equal or less than the PowerShellMaxCmdlets parameter as an Exchange cmdlet will also count as a PowerShell cmdlet;
• PowerShellMaxCmdletsTimePeriod – defines the time period used by PowerShellMaxCmdlets and ExchangeMaxCmdlets in seconds. Be sure to set this parameter at the same time as you set any of the above two!

If, for example, PowerShellMaxCmdletsTimePeriod was set to 60 and PowerShellMaxCmdlets to 2, the message a user would receive when he/she tries to run more than 2 cmdlets in less than 60 seconds is shown below:

Figure 1: Throttling Policy “Back-Off” Message

Please note, however, that only one PowerShell policy is configured by default as you will see in Figure 2.

Let’s now have a look at the Default Throttling Policy and how we can create or edit these policies.

Default Policy

When you first create an Exchange organization, a default throttling policy is created which applies to all users within the organization. With Exchange 2010 RTM, only <Component Acronym>MaxConcurrency parameters were enabled by default, but with SP1 all throttling policies are enabled by default.

Normally the default policy is sufficient for most environments. However, organizations might have specific requirements which can only be met by changing this policy or by creating a new one for every user or for a specific group of users. For example, if you have Blackberry Enterprise Server [BES] in your organization, Research In Motion [RIM] recommends that a new throttling policy gets created so that the account used by BES is not throttled at all.

The picture below shows a Default Throttling Policy created in Exchange 2010 SP1:

Figure 2: Default Throttling Policy

Managing Policies

Throttling Policies are simply AD objects saved under:

CN=Global Settings, CN=Exchange Org, CN=Microsoft Exchange, CN=Services, CN=Configuration, DC=Domain, DC=COM:

Figure 3: Throttling Policy Location in AD

If you check its properties, you will be able to see all the values from Figure 2:

Figure 4: Default Throttling Policy AD Attributes

At the time of writing this article where the latest version of Exchange 2010 is SP2, throttling policies can only be managed using the Exchange Management Shell [EMS] and the New-ThrottlingPolicy, Get-ThrottlingPolicy, Set-ThrottlingPolicy, Remove-ThrottlingPolicy, Get-ThrottlingPolicyAssociation and Set-ThrottlingPolicyAssociation cmdlets.

Before we start configuring Throttling Policies, it is important to understand how these are associated with users, which can be done using two methods:

1. Using the Set-Mailbox cmdlet to associate a Throttling Policy with a single user mailbox or a group of users mailboxes;
2. Using the Set-ThrottlingPolicyAssociation cmdlet to associate a Throttling Policy with a specific object, such as a user with or without a mailbox, a contact or a computer account.

In the beginning of this article I stated that the default policy is automatically enabled and associated with every user account. So let’s confirm this by running Get-Mailbox and checking for the ThrottlingPolicy parameter:

Figure 5: Checking Users’ ThrottlingPolicy Attribute

But the ThrottlingPolicy parameter is null! Shouldn’t it mention the default policy? In my opinion it should, but this actually means that the default policy is applied to the user… Using the Get-ThrottlingPolicyAssociation produces the same outcome, with ThrottlingPolicyId also blank:

Figure 6: Checking Throttling Policy Association

Now let’s create a new Throttling Policy for BES for example. Because the default Throttling Policy may affect the performance of BES if you have more than a few hundred BES users, RIM strongly recommends administrators to turn off client throttling for the Windows account used by BES.

To do this, we use the New-ThrottlingPolicy cmdlet to set the required components to $null which means unthrottled:

New-ThrottlingPolicy BESPolicy -RCAMaxConcurrency $null -RCAPercentTimeInAD $null -RCAPercentTimeInCAS $null -RCAPercentTimeInMailboxRPC $null -EWSMaxConcurrency $null -EWSPercentTimeInAD $null -EWSPercentTimeInCAS $null -EWSPercentTimeInMailboxRPC $null -EWSMaxSubscriptions $null -EWSFastSearchTimeoutInSeconds $null -EWSFindCountLimit $null

Note 1:
Don’t set throttling policy parameters to $null unless you have a very good reason to do so and fully understand its consequences!

Note 2:
When creating a new policy, any parameter you do not configure will inherit its value from the default policy.

Now that we have our policy created, we need to associate it with the account used by BES. As mentioned before, there are two methods we can use but both produce the same results, so let’s use the easiest one and confirm that the policy was actually associated with the user account:

Set-Mailbox BESAdmin -ThrottlingPolicy BESPolicy

Figure 7: Checking BES Throttling Policy Association

Once this new policy kicks in, usually in 15 minutes, the BESAdmin account will no longer be throttled by the default policy but by the new BESPolicy throttling policy.

Common Issues with Throttling Policies

From my experience so far, there are two main issues caused by Throttling Policies: one is an inadequate policy for BES, which we already addressed; and the second is to do with mobile devices.

If we look at the settings for ActiveSync on the default policy:

Figure 8: EAS Settings on Default Throttling Policy

We see that by default users can only have up to 10 devices connected at the same time, but also they can only have a maximum of 10 devices at any point in time! Although the first one is more than fair, the second might cause issues for administrators constantly testing several devices for example. When EASMaxDevices is reached, users will receive a similar message to the one below on their mailboxes (not on the device!) informing them why they can’t connect their mobile device (in this example I changed the limit to 1 as I don’t have 10 devices for testing):

Figure 9: EAS Partnership Error

To overcome this limitation, we can:

1. Increase EASMaxDevices on the Default Throttling Policy;
2. Create a new policy just for users who need to connect more than 10 devices
3. Login to Outlook Web Access, go to Options -> Phone -> Mobile Phones and delete any unused partnerships to reduce the number of devices associated with the account.

Conclusion

The great improvement of Client Throttling Policies in Exchange 2010 is part of Microsoft’s commitment to improve the performance of Exchange and provide administrators with a more granular level of control. It is a very good improvement but it can also cause some administrative headaches if administrators are not aware of them and what to look for when problems occur.