Exchange 2010 - High CPU utilization for the RpcClientAccess service

Within the past couple of weeks I have witnessed multiple instances where the RpcClientAccess service is experiencing high CPU utilization on the CAS server and in some instances also on a public folder server. I captured performance data from these systems to get a better picture of what is happening on the server. The analysis of this data does not show any unusual activity other than this process utilizing more CPU than normal. This tells me that the process itself must be doing something that is causing these extra cycles.

To determine what is happening within the RPC Client Access service we need to review the Rpc Client Access logs on the server. While parsing thru this log file I found a high volume of PublicLogon requests coming into this CAS server for a public folder database. These logon attempts are typical behavior as the CAS server will then redirect these requests to the mailbox server hosting the database. But taking a closer look at these requests and I see the database is last mounted on the expected mailbox server, however the client is being redirected to another server. (If you only have one public folder database you will simply see a high volume of requests coming into you CAS server and the CPU will most likely spike to near 100 percent). The log entry looks like the following:

2012-02-15T14:00:51.707Z,6,1,/o=EXCHORG/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=Exchange User,,MFCMapi.exe,14.0.6109.5000,Classic,,,ncacn_http,,PublicLogon,1144 (rop::WrongServer),00:00:00.0621272,"Logon: Public, in database c83d21d5-7228-455e-978d-81d6649863a1 last mounted on MBX1.contoso.com at 1/28/2012 8:01:35 PM, currently Mounted; Redirected: alternate server requested, suggested new server: /o=EXCHORG/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=MBX2",RopHandler: Logon:

This issue is caused because the RpcClientAccess service on the mailbox server hosting the public folder database is not responding to client connections. If we take a look at the RPC Client Access logs on this server we should see minimal activity. This is a known issue that is documented in KB 2535105 (Outlook client applications cannot connect to public folders after you install Exchange Server 2010 SP1). The RPC Client Access service acts as a proxy for public folder availability requests and these requests are not completed. Eventually new connections for these EWS calls cannot be made because the connection group limit is reached. The KB article provides a registry key that is available starting with SP1 RU4 that disables these available service calls by the RPC Client Access service.

A .NET hotfix that resolves the underlying issue has been released since the posting of the Exchange KB, KB2497453 (FIX: Application may stop responding after several HTTP requests are aborted if you use the HttpWebRequest class to send requests in the .NET Framework). Install this hotfix on the mailbox server hosting the public folder database and reboot the mailbox server. Once this server is back online we need to restart the Rpc Client Access service on the CAS server to clear the cache. Then when we review the Rpc Client Access logs on the CAS server we should see clients being redirected to the server where the public folder database is mounted.