Troubleshooting the Exchange Recipient Update Service (RUS)
The Recipient Update Service (RUS) is the component of Exchange that is responsible for generating mail proxy addresses for all mail-enabled objects in an Exchange organization. Normally this component runs in the background and requires little configuration or maintenance. There are times when issues do occur and fixing them should be a top priority in order to keep mail flowing. Let’s take a look at what the RUS does before going into some common troubleshooting steps:
Description of RUS
When you perform the initial install of Exchange, the Recipient Update Service is installed and a default recipient policy is created. This policy is responsible for ensuring that all mail-enabled objects in the Exchange organization have a valid SMTP address following the [email protected] naming format. You can create a new policy that can be configured to create each SMTP address following a different naming convention such as [email protected] Microsoft has a list of best practices to follow when creating and/or editing recipient policies.
- Create a new recipient policy and assign it a higher precedence rather than editing the default policy
- Keep the number of recipient policies to a minimum
- Rebuild the RUS with caution
A lack of understanding of the RUS is the major cause of issues. Often administrators apply a policy without understanding what will be changed. Exchange does not provide much warning about the impact a change will make. On top of that, organizations using a 3rd party application to create and assign SMTP addresses, through MIIS for example, can cause further damage by applying recipient policies blindly. So what do we do when RUS takes a vacation?
Troubleshooting Common Issues with RUS
As previously mentioned the Recipient Update Service runs quietly in the background and requires little or no maintenance. When issues do occur there are three basic steps to troubleshooting RUS.
- Enable Diagnostic Logging
- Choose an object, or objects to monitor
- View the Application Log for errors
To begin troubleshooting RUS we first determine if we have more than one recipient policy, if so, set the schedule for all but one to Never Run. In the case of multiple policies, you may be required to go back and enable another policy if you find nothing wrong with the first. Just ensure that only one policy is scheduled to run at a time.
With the schedule configured, the next step is to enable Diagnostic Logging and set it to log the maximum amount of events on the following services and objects.
MSExchangeAL – LDAP Operations
MSExchangeAL – Address List Synchronization
MSExchangeSA – Proxy Generation
To do this, open up Exchange System Manager (ESM) and drill down to Administrative Groups | Servers | Servername, right click the server you wish to configure logging on and select Properties and then go to the Diagnostics Logging tab. Under Services, select MSExchangeAL and set LDAP Operations and Address List Synchronization to maximum (see Figure 1). Next select the MSExchangeSA services and set Proxy Generation to maximum. (Note: Proxy Generation is not available on Exchange 2000 servers). Finally create a test object for the RUS to stamp.
Figure 1: Diagnostic Logging
Verify RUS is Running
With Diagnostic Logging enabled, wait a few minutes and you should see two events show up in the Application event log with IDs 8011 and 8012. These events verify that RUS has started. If you do not get these messages, restart the Microsoft Exchange System Attendant service. Once this service is started you will see a number of new events logged, the first of which, 9006 and 9008, notify you that Abv_dg.dll is loading and then starting.
If event ID 9006 appears, but you never get event ID 9008, you are performing this task on a front-end server. On a front-end server Abv_dg.dll does not exist and RUS must be run on a back-end server.
Verify RUS Queries
If event ID 8011 and 8012 do appear in the Application event log, the next step is to determine if RUS has queried for any changes, for this you will require ADSIEdit. Open up ADSIEdit and connect to the Domain Controller that RUS is pointing to. Drill down to Domain NC | DC=domain,DC=com | CN=Users (or wherever you created your test object) and right click the test object selecting Properties. Locate the uSNChanged value and record it. Next open up the Event Viewer on the Exchange server you have enabled logging on and search for
Event ID: 8011
Description: Base DC
When the search is done, open the first event in the list that will include information about the last search completed by the RUS.
Locate the line in the event (USNChanged>=########)(uSNChanged<=########) and determine if the value you recorded for the test object falls into this range (see Figure 2).
Figure 2: USNChanged
Based on this information we can determine a few things:
- If the uSNChanged attribute is higher that the range shown in the event, RUS has not queried this object yet. This is common when you have selected to rebuild the RUS which, depending on the size of the domain can take anywhere from a few minutes to a few days. When you initiate a Rebuild, the RUS starts from scratch with a uSNChanged value of 1 and queries all objects in the domain.
- If uSNChanged attribute value for the test object falls below the range in the event RUS has passed this object. Open up the next oldest event with ID 8011 until you find the event that contains a range that the value falls into. If you have trouble finding it you can modify the test object and wait for it to be queried again and find the event for it.
- If the Application log does not contain an event ID 8011 that has "Base 'DC" in the event description, the domain RUS has not started processing yet or the event was over written by more recent events. A rebuild operation can cause this as it can generate a large number of these events.
You can determine if a rebuild operation is taking place by lowering Diagnostics Logging on all items except the Address List Synchronization object and then filter event 8329, which specifies the start of a rebuild operation. At intervals of 10% event ID 8332 will be logged and when the rebuild is complete event ID 8330 will be recorded. If you see event 8329 and 8332, but no 8330 then the rebuild is still running and you should wait until it is complete.
Verify RUS Query Results
At this point we should find event ID 8011 that will inform us that the search was performed on a range of USNs, which includes the USN of your test object. Now we can see whether this returned any results. Events 8012 and 8169 will give us the results in the description.
Event Type: Information
Event Source: MSExchangeAL
Event Category: LDAP Operations
Event ID: 8012
Time: 7:56:52 AM
Description: Search of directory DC-01.tlaconsulting.ca at base 'DC=tlaconsulting,DC=ca' returned 3 objects.
Event Type: Information
Event Source: MSExchangeAL
Event Category: Address List Synchronization
Event ID: 8169
Time: 7:53:01 AM
Description: Retrieved all directory changes under: 'DC=tlaconsulting,DC=ca'.
There are a few common scenarios that apply to Event 8012.
- If there is no 8012 event generated after an 8011 event occurs, Exchange did not receive a response to the search. This is usually indicative of a network issue where the Exchange server cannot contact Active Directory.
- When the search returns zero objects (and you are certain there should be some); this usually indicates a permissions error. In this case the Exchange server computer account does not have the correct permissions required to view user objects. In order to view user objects, the Exchange server needs to be part of the Exchange Enterprise Servers group.
- If more than 20 objects are found you will see multiple 8012 events, as results are returned in groups of 20. You should see one 8012 event for every 20 objects that the query returned.
The Recipient Update Service is a fundamental component of Exchange and ensuring it is up and running is crucial to a properly functioning Exchange envirmoment. Hopefully these steps help you determine the cause and aid in finding a resolution in the unfortunate event that something does go wrong.
The Recipient Update Service Uncovered