Exchange 2013 - The case of the mysterious crashing RPC Proxy Application Pool

We have seen some issues lately where application pools have been crashing. Usually this is a brief service interruption and sometimes the users do not notice there was an issue. There have been a few cases recently where we are seeing the MSExchangeRPCProxyAppPool crashing. Usually you will see the following events in the System log

 Log Name: System
Source: Microsoft-Windows-WAS
Date: 11/8/2013 9:16:32 AM
Event ID: 5011
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
A process serving application pool 'MSExchangeRpcProxyAppPool' suffered a fatal communication error with the Windows Process Activation Service. The process id was '4356'. The data field contains the error number.


Log Name: System
Source: Service Control Manager
Date: 11/8/2013 9:16:53 AM
Event ID: 7031
Task Category: None
Level: Error
Keywords: Classic
User: N/A
The Microsoft Exchange RPC Client Access service terminated unexpectedly. It has done this 1 time(s). The following corrective action will be taken in 5000 milliseconds: Restart the service.

Microsoft determined that in most cases this was an issue and have provided a fix in Exchange 2013 CU3. For most people, this resolves their issue.

However, there is another reason that this could be happening, so if you have applied Exchange 2013 CU3 and you are still hitting this issue, please read on.

In 2013 we move from a Public Folder database to a Public Folder Mailbox. We provide step by step migration instructions here

Sometimes when you delete things, AD doesn't always get the update that it should. We have seen this in the past with connectors or user objects where we see reference to a deleted object. That is what can be happening here. In some of the cases that I have worked, the Legacy Public Folder is showing as a deleted object in AD, specifically the msExchHomePublicMDB attribute. To check to see if this is populated, take the following steps

1. Open ADSIEdit

2. Connect to the Configuration container

3. Expand Services

4. Expand Microsoft Exchange

5. Expand your Organization name

6. Expand Administrative Groups

7. Highlight Databases

8. In the right hand pane you will see a list of databases. 

9. Right click on the database object and chose Properties

10. Scroll down to msExchHomePublicMDB

11. If you see reference to the old database delete the value

12. Check the rest of the databases to make sure that they are not populated in the same way.

Once this has been removed, you should stop seeing the System events noted above.

Thanks to Carl McDaniel, Corey Callaway and JT Thompson for their valuable assistance with this.