Errors Connecting to the Data Warehouse (Health Service Module Events 31557, 31561, 31563, 31569)

  • Article

Today I worked with a user...a smart user.  He told me a story I'd heard before - "I didn't touch anything but SCOM broke!" - and I have to say this time I believed him.... 

Basically out of the blue, his management servers (MS) were unable to talk to the DW.  And when we looked thru the events below, we noticed that the MS were trying to use the SCOM Action Account instead of the Data Warehouse action accounts.  The OpsDB appeared fine, the console ran, and agents were communicating with the management group.

Errors seen on the Management Servers

Log Name:      Operations Manager
Source:        Health Service Modules
Event ID:      31557
Task Category: Data Warehouse
Level:         Error
Description:  Failed to obtain synchronization process state information from Data Warehouse database. The operation will be retried.
Exception 'SqlException': Management Group with id ''xxxx'' is not allowed to access Data Warehouse under login ''[ID]''

One or more workflows were affected by this. 

Workflow name: Microsoft.SystemCenter.DataWarehouse.Synchronization.MaintenanceMode
Instance name: Data Warehouse Synchronization Service
Instance ID: {xxx}
Management group: [name]

+++++++++++++++++
Log Name:      Operations Manager
Source:        Health Service Modules
Event ID:      31561
Task Category: Data Warehouse
Level:         Error
Description:  Failed to enumerate (discover) Data Warehouse objects and relationships among them. The operation will be retried.
Exception 'SqlException': Management Group with id ''xxxx'' is not allowed to access Data Warehouse under login ''[name]''

One or more workflows were affected by this. 

Workflow name: Microsoft.SystemCenter.DataWarehouse.Discovery.StandardDataSet
Instance name: Alert data set
Instance ID: {xxx}
Management group: [name]

+++++++++++++++++

Log Name:      Operations Manager
Source:        Health Service Modules
Event ID:      31563
Task Category: Data Warehouse
Level:         Error
Description:  Failed to enumerate Data Warehouse components for deployment. The operation will be retried.
Exception 'SqlException': Management Group with id 'xxxx'' is not allowed to access Data Warehouse under login ''[name]''

One or more workflows were affected by this. 

Workflow name: Microsoft.SystemCenter.DataWarehouse.Deployment.Component
Instance name: Data Warehouse Synchronization Service
Instance ID: {xxx}
Management group: [name]

+++++++++++++++++
Log Name:      Operations Manager
Source:        Health Service Modules
Event ID:      31569
Task Category: Data Warehouse
Level:         Error
Description:  Report deployment process failed to request management pack list from Data Warehouse. The operation will be retried.
Exception 'SqlException': Management Group with id ''xxxx'' is not allowed to access Data Warehouse under login ''[name]''

One or more workflows were affected by this. 

Workflow name: Microsoft.SystemCenter.DataWarehouse.Deployment.Report
Instance name: Data Warehouse Synchronization Service
Instance ID: {xxx}
Management group: [name]

 

So what's going on?

A little searching pulled up a blog by Stefan Roth - https://stefanroth.net/2012/12/30/scom-2012-event-id-31557-health-service-modules/#comment-26259.   I won't duplicate his post, but the gist is that the Data Warehouse profiles had no action accounts associated with them.  The solution is to add them back in.  Here's what the profiles should look like (at least in my lab).  Note: They have "account" in their names but are actually profiles!

Data Warehouse Account (Profile)

Data Warehouse Report Deployment Account (Profile)

How did this happen?

As of now, I'm still investigating...