Top Three Configuration Issues with Exchange Management Pack
You can use the Exchange Server 2003 Management Pack to monitor the performance, availability, and security of Microsoft® Exchange Server 2003, alerting you to events that have a direct impact on server availability, while filtering out events that require no action. When it is imported, Exchange Management Pack begins to monitor the servers that are running Exchange in your environment. However, some scripts require configuration. Sometimes, during the configuration, you may get some errors that you aren’t sure how to troubleshoot. This article discusses three of the most common issues with configuration that Microsoft customers call us for help with resolving. Before we talk about those issues, let's review just how a server that runs Exchange is prepared for monitoring.
How a Server is Prepared for Monitoring
When you deploy the Exchange Management Pack, Microsoft Operations Manager (MOM) takes care of most of the configuration for you, working under the hood to identify servers that are running Exchange in your environment and to push the rules included in the management pack to those servers. Some of these rules check the current configuration of the server for certain attributes and may call other rules that make the changes necessary to prepare the server for monitoring. When these rules begin to fire, you will see events bubbling up in the MOM Administrator Console that indicate this process is underway. Eventually, you should see other events directing you to take the steps necessary to finish the configuration work that MOM began.
At this point, you would run the Exchange Management Pack Configuration Wizard to help you make the final configuration changes that MOM requires to fully monitor Exchange functionality such as mailbox availability, mail flow, or services. Unfortunately, you may also discover that you can’t run the wizard because there has been a failure in the preliminary preparation that MOM has been doing.
Issue 1: Missing 9986 Events in MOM
One problem we see from time to time in Exchange Management Pack support is the inability to finish the configuration for monitoring of a server that is running Exchange. This situation can be due to missing data that is normally published for you behind the scenes when the management pack is deployed. We’ll try to explain why this may happen, and what to look for if it does.
From the configuration wizard, you may see the following error:
Error: Cannot configure the mailbox access account on computer <servername>. This configuration can only be made after the Exchange MOM event 9986 is registered by MOM.
We'll explain this 9986 event and when you should expect it, and some reasons why you may not see it. One of the first rules that fires on a backend Exchange server is the mailbox availability test. The script associated with this rule checks for the registry data that makes the test possible. The HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExMPLS key is used to store the encrypted credentials of the domain account that is used to access monitoring mailboxes on each server that is running Exchange. This Mailbox Access Account (MAA) must have permissions to log on to the test mailboxes because MOM uses these credentials to impersonate the account during logon. If the registry data is missing, this rule generates an event that triggers another rule that in turn initiates the publication of the missing key used to store the credentials.
To publish the key, Exchange servers need “helper” Distributed COM (DCOM) objects installed to work in conjunction with the publishing rules. The DCOM object responsible for the publication of the registry key that stores the MAA credentials is installed as part of Exchange Server 2003 setup. However, for earlier versions of Exchange, MOM installs the helper objects as needed, or they can be installed manually. The publishing rule calls the DCOM object to publish the required data on the server.
If DCOM and the helper object function are working as expected, the ExMPLS registry key is created. When the data is published, the 9986 event is generated and you should be able to run the configuration wizard against that server. The wizard then writes the MAA credentials in encrypted form to the key. And, if you are afraid of missing the event, don’t worry, there is a rule that runs daily that checks and generates the 9986 event if the data has been successfully published before.
If publication fails, the rule generates events and they appear in the MOM Administrator Console to alert the administrator of failure. Failures can be caused by a missing or malfunctioning DCOM object, or misconfigured DCOM permissions. Missing or malfunctioning DCOM objects installed as part of the Exchange Management Pack can be difficult to troubleshoot and resolve. Contact Microsoft Customer Support for assistance if you experience a failure to generate the 9986 event. For more information about investigating and resolving problems with default DCOM permission settings, see Microsoft Knowledge Base article 274696, Actions such as search and drag and drop do not work because the default access permissions have been changed in the Dcomcnfg.exe tool.
Issue 2: MAPI Logon Verification Script Errors
After you have generated the 9986 event, you can run the Configuration Wizard to configure your environment to monitor Exchange. The Configuration Wizard creates the Mailbox Access Account (MAA) and the Test (Agent) Accounts and mailboxes. These accounts are used for the MAPI logon script and OWA logon script to monitor the Exchange servers.
To verify that the Mailbox Store is mounted and that Microsoft Office Outlook® users can successfully log on, the Exchange Management Pack runs the MAPI Logon Verification script. In this script, the MAA credentials are used to actually log on and open the mailbox of the Test mailbox (servernameMOM). The Mail Flow Verification scripts also use MAPI logon to send and receive messages. You can see the logon to the server that is running Exchange by looking in Exchange System Manager under the Logons section under the Mailbox Store folder. You should see the Test mailbox; note that the MAA was the Microsoft Windows® account that was used to log on.
MAPI logon script errors are some of the most frequent issues that we see. You might encounter several different error messages, including MAPI_E_NOT_FOUND, MAPI_E_LOGON_FAILED, and MAPI_E_NOT_INITIALIZED. For more information about some of the common causes of these error messages, see "Permissions and Directory Access Errors" in the Exchange Server Management Pack Guide for MOM 2005.
If you are getting MAPI logon errors, follow all of the the troubleshooting steps in this section because there can be multiple causes for these issues and resolving one can unmask the underlying issue. For more information about MAPI errors, see Knowledge Base article 238119, INFO: List of Extended MAPI Numeric Result Codes.
Additionally, you should verify that the MAA has full mailbox rights on the mailbox used for the MAPI Logon test. You can verify if the MAA has permission by opening Outlook as the Mailbox Access account. Then, open the Test mailbox by going to File, Open, Other User’s Folder, and selecting the test mailbox. You should be able to see the Inbox of the Test account. If you cannot open the mailbox, open Active Directory Users and Computers and look at the properties of the Test mailbox. Click Exchange Advanced, and then click Mailbox Rights. The MAA should be listed here, and it should be granted Full Mailbox Access, Delete mailbox storage, and Read permissions.
If you are still having problems opening the Test mailbox, check to be sure that none of the mailboxes that MOM uses (MAA and Test mailboxes) are hidden. If the accounts are hidden in the Global Address List (GAL), the MAPI Logon script or Mail Flow Verification scripts will be unable to use the accounts. If the accounts are not hidden but you are still unable to log on to Outlook, sometimes it is helpful to determine how the accounts were created. The manner that the accounts were created such as by using the Configuration Wizard, manually creating accounts, or creating accounts using a third-party provisioning product may give you clues as to why you cannot log on to Outlook.
As another test, log on to Windows on the server that is running Exchange using the logon credentials of the Mailbox Access Account. Open Exchange System Manager, and expand Administrative Groups, expand Servers, and continue expanding until you can choose Mailboxes on the left side. Can you see the list of mailboxes on the right side? If not, the Mailbox Access Account may have been denied View Information Store Status. You can follow the ADSI Edit procedure below, and drill down to the properties of the server. Click the Security tab (see step 10 below). Select the Mailbox Access account, and scroll down in the list of permissions. Make sure that View Information Store Status has an inherited Allow. If it has an inherited Deny, check the properties of CN=Servers, CN=yourAdministrativeGroup, and CN=Administrative Groups to locate the Deny.
Here are the steps in ADSI Edit to check for the explicit Deny:
- Open ADSI Edit.
- In the left pane, under Console Root, right-click ADSI Edit.
- Choose Connect to.
- In the Connection Settings window, look in the Connection Point section, and choose Select a well known Naming Context, and then in the drop-down menu, choose Configuration. Click OK.
- In the Console Root window, expand the plus sign beside Configuration.
- Expand the plus sign beside CN=Configuration, DC=.
- Expand CN=Services.
- Expand CN=Microsoft Exchange.
- Highlight CN=<yourOrgName>, and then click Properties.
- Click the Security tab.
- Locate the Mailbox Access account, and verify that it does not have Send as or Receive as permissioned with a Deny.
While you are in ADSI Edit, verify that Send as and Receive as are not denied at the Administrative Group level, or on the Exchange server.
Active Directory® directory service problems can also result in intermittent failure of the MAPI Logon Verification script. MAPI logon fails if it cannot access a domain controller, or the domain controller does not respond in a timely manner. Start the Exchange System Attendant service if it is not started. Verify the configuration for the agent mailboxes, and correct any errors in configuration. Verify that the domain controllers in the domain are accessible, and that users can log on using Outlook.
You can also enable logging on the server that is running Exchange to help troubleshoot the problem. On the server that is running Exchange, add the following registry key (type DWORD) and set the value to 1:
Remember to set the value to 0 when you want to turn off logging.
Stop and restart the MOM service on the server that is running Exchange. Make sure to grant the Mailbox Access Account full rights to %systemdrive%; otherwise, nothing useful will be logged to ExMPLS_LOG.txt. Be sure to remove the right afterward!
Wait until the MAPI logon verification script runs (wait overnight), and look for a file called ExMPLS_LOG.txt that is located in the %systemdrive%. This log file is often useful to troubleshoot MAPI logon issues.
If you are still having problems with MAPI logon after going through these steps, call Microsoft Customer Service and Support (CSS) for further troubleshooting.
Issue 3: Outlook Web Access Monitoring
Problems with the Microsoft® Office Outlook® Web Access 2003 and Microsoft Outlook® Mobile Access logon verification scripts are another issue that our customers sometimes see. The Exchange Management Pack provides several rule groups under the Availability and State Monitoring Rule Group that encompasses synthetic logon testing. These rule groups simulate a mailbox logon using one of our mobility technologies such as Outlook Web Access, Outlook Mobile Access, or Exchange ActiveSync. Due to the complexity of the synthetic logon process, we sometimes see errors reported in MOM indicating that the logon failed even though it may actually be possible to log on to Outlook Web Access by means of Internet Explorer. One such error we see often that falls into this category is the following:
OWA Logon Failed.
Cannot measure OWA availability. Unexpected error.
No Exchange virtual servers and virtual directory (SSL enabled) can be found on this server to form a valid url. Try providing the url in the custom urls registry key (see Configuration guide for detail)
At first glance, it appears that the logon was unable to find the Exchange virtual server. Therefore, logon failed. However, you know you have an Exchange virtual server. And, you use Outlook Web Access everyday to check your mail when you're out of the office. So, what is this error actually telling us? The subtle clue in the above error is the URL that is returned, Undefined. To better understand what’s happening here, let's take a few minutes to explain how the Outlook Web Access synthetic logon process works.
The Outlook Web Access Logon Verification script initiates the process. Although the script has many tasks, its core roles are as follows:
- Enables a reference to the OWAAvailability automation object which is returned to objOwa
- Reads the following registry key and retrieves the values of the CustomURL and BEAccount strings: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange MOM\FEMonitoring\<FEServerName>
- Passes the mailbox name, and any customURL values found, to objOwa.OWALogon
- Logs results (success or failure) in the MOM Operator console
The OWALogon script function takes three parameters: the BEMailbox that we want to log on to, a timeout value, and the URL. This function can be called multiple times, dependant upon how many customURLs are found in the registry. The first call is always made with a NULL URL value that will ultimately test https://localhost, and subsequent calls are made in a loop testing the customURLs. We are interested in the first call with the NULL URL.
The OWALogon script function is responsible for passing its received arguments to the automation object objOwa.OWALogon, which is going to be doing the grunt work for us. Because we did not pass in all three arguments (leaving the URL NULL), the first thing that objOwa.OWALogon must do is call a method that will construct a URL for us to test. This method, GetExchangeURL, will return the URL of the specified mailbox. However, when GetExchangeURL is unable to determine the URL of the passed-in mailbox, it returns NULL, logging our unexpected error.
Now that we know why the error is logged, you can see why we suggested that the URL:undefined line was a subtle clue. As the error alluded to, we were unable to determine what the URL is; therefore, we failed. Time to unravel GetExchangeURL.
The first time we reviewed this code, we were shocked at how much work the developers put into this method. Really, all we have to do is return https://localhost, right? Well, that depends. What if we have multiple Exchange virtual directories or servers, each published to their own URL (think hosting)? We need to know which virtual server the mailbox belongs to so we know that we're testing the correct URL. Okay, so we can’t make any assumptions and need to be able to construct the correct URL for the test mailbox. We don’t have room here to describe the entire GetExchangeURL process in detail; however, we will mention briefly what the major steps are.
Get the primary SMTP address of the test mailboxes This is a simple Lightweight Directory Access Protocol (LDAP) search for the test mailbox using samAccountName. The proxyaddress attribute is then read and parsed, looking for the address starting with SMTP: (note the caps; non-primary addresses will be lowercase). This value is then parsed to pull the domain name. To test this step, perform the following LDAP search and verify that your test mailbox is returned:
Search for the virtual directory More LDAP searches. We won’t go into too much detail here because it’s pretty generic and uninteresting. The end result is a query for all HTTP virtual servers on the front-end server.
Check the virtual directory SMTP domain Check the virtual server for the presence of the msExchDefaultDomain attribute, and verify if it’s set (by default, it’s not). If so, see if it matches the mailbox’s primary or non-primary SMTP domains. If msExchDefaultDomain is not set, see if the mailbox’s primary SMTP address matches the default recipient policy’s primary SMTP domain. For example, if the test mailbox was stamped by a custom policy using @it.contoso.com, but the default recipient policy is configured to stamp @contoso.com, we will fail.
This situation is the most common cause of GetExchangeURL returning NULL that we see in Microsoft CSS.
Verify SSL is enabled on virtual directory Check for the presence of the IIS://localhost/W3SVC/1/root/Exchange/AccessSSLFlags metabase property. You can check this value using an Internet Information Services (IIS) Resource Kit tool called Metabase Explorer. For more information about this tool and others in the IIS Resource Kit, see Knowledge Base article 840671, The IIS 6.0 Resource Kit Tools. For more information about the AccessSSLFlags property, see AccessSSLFlags Metabase Property (IIS 6.0) in the IIS 6.0 Reference section of the IIS 6.0 Operations Guide.
Check the value of secureBindings Get the IIS://localhost/W3SVC/1/secureBindings metabase property, and parse it. If the secureBindings property starts with a :<portNumber> (:443), the virtual server is set to “All Unassigned”, so then the URL is set to https://localhost. If the property does not start with a “:”, parse out the IP address that has been assigned to this port. This IP address is then verified using a WMI call to Win32_NetworkAdapterConfiguration, and the IPAddress property that is returned is compared to the IP address found in the secureBindings property. If a match is found, set the URL to https://ipaddress. If no match is found, NULL is returned. Again, you can use Metabase Explorer to see what the property is set to. If it starts with an IP address, you can use WBEMTest to query the Win32_NetworkAdapterConfiguration class and check the IPAddress property. For more information about the secureBindings property, see SecureBindings Metabase Property (IIS 6.0) in the IIS 6.0 Reference section of the IIS 6.0 Operations Guide. For more information about how to use WBEMTest, see Scripting Eye for the GUI Guy.
If after checking the above, you are still unable to determine why GetExchangeURL has returned NULL, Microsoft CSS has a tool called OWACheck that can be immensely helpful in troubleshooting this issue. Please contact CSS for this tool and further troubleshooting assistance.
For More Information
For more information, see the following resources: