ConsolidatorType

Article
05/16/2012

Applies To: Operations Manager 2007 R2

ConsolidatorType complex data type is used to consolidate multiple incoming data items based on a specific schedule or time interval.

Schema Definition

<xsd:complexType name="ConsolidatorType">
  <xsd:sequence>
    <xsd:element name="ConsolidationProperties">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="PropertyXPathQuery" type="xsd:string" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
    <xsd:element name="TimeXPathQuery" type="xsd:string" minOccurs="0" maxOccurs="1"/>
    <xsd:element name="StoreState" type="xsd:boolean" minOccurs="0" maxOccurs="1"/>
    <xsd:element name="TimeControl">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="Latency" type="xsd:int" default="60" minOccurs="0" maxOccurs="1"/>
          <xsd:element name="DrainWait" type="xsd:int" default="120" minOccurs="0" maxOccurs="1"/>
          <xsd:choice>
            <xsd:element name="GenericSchedule" type="PublicSchedulerType"/>
            <xsd:element name="WithinTimeSchedule">
              <xsd:complexType>
                <xsd:sequence>
                  <xsd:element name="Interval" type="xsd:int"/>
                </xsd:sequence>
              </xsd:complexType>
            </xsd:element>
          </xsd:choice>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
    <xsd:element name="CountingCondition">
      <xsd:complexType>
        <xsd:sequence>
          <xsd:element name="Count" type="xsd:int" minOccurs="0" maxOccurs="1"/>
          <xsd:element name="CountMode">
            <xsd:simpleType>
              <xsd:restriction base="xsd:string">
                <xsd:enumeration value="OnNewItemNOP_OnTimerOutputRestart">
                  <xsd:annotation>
                    <xsd:documentation>OnNewItem:   NOP. OnTimer:   output/restart</xsd:documentation>
                  </xsd:annotation>
                </xsd:enumeration>
                <xsd:enumeration value="OnNewItemTestOutputRestart_OnTimerSlideByOne">
                  <xsd:annotation>
                    <xsd:documentation><![CDATA[OnNewItem:   if (x>=Count) output/restart. OnTimer:   Slide1]]></xsd:documentation>
                  </xsd:annotation>
                </xsd:enumeration>
                <xsd:enumeration value="OnNewItemTestOutputRestart_OnTimerRestart">
                  <xsd:annotation>
                    <xsd:documentation><![CDATA[OnNewItem:   if (x>=Count) output/restart. OnTimer:   Restart]]></xsd:documentation>
                  </xsd:annotation>
                </xsd:enumeration>
              </xsd:restriction>
            </xsd:simpleType>
          </xsd:element>
        </xsd:sequence>
      </xsd:complexType>
    </xsd:element>
  </xsd:sequence>
</xsd:complexType>

Remarks

The ConsolidatorType data type contains the following sub-elements:

Parameter	Type	Description
ConsolidationProperties	Complex	Contains a set of XPath expressions specifying like data items to be correlated. This element is required but can be empty.
TimeXPathQuery	String	Optional. Specifies a data item property to use as the time value other than the actual time value on the data item.
TimeControl	Complex	Defines the time window(s) during which events will be consolidated.
CountingCondition	Complex	Defines the method of counting data items for consolidation.

ConsolidationProperties

This element contains one or more XPath expressions that identify the criteria for the incoming data items to be consolidated. This is not required if an expression filter module is used (before the consolidator module) to filter the data items to a set of items that should be considered the same.

This element is always required, but it can be left empty when all incoming data items should be consolidated as one. For example:

<ConsolidationProperties/>

If there are multiple incoming data items that you want to consolidate into groups of items, then specify the data item properties that will be used to classify the data items as the same for consolidation purposes. The ConsolidationProperties element is made up of zero or more PropertyXPathQuery elements. The exact expression is dependent on the data item that is the input to the consolidation module.

For example, if you are monitoring the Windows event log, you can use the event provider data source to collect all events with event IDs in the range 101 to 110. However, you might want to output consolidated data only if there are more than 10 events of any one of these event IDs within 60 seconds. In this scenario, you specify the following configuration:

<ConsolidationProperties>
  <PropertyXPathQuery>EventDisplayNumber</PropertyXPathQuery>
</ConsolidationProperties>

You can specify multiple properties that must match to consolidate the data items. The following code example shows how to use the first event parameter and the event ID to group like events together:

<ConsolidationProperties>
  <PropertyXPathQuery>EventDisplayNumber</PropertyXPathQuery>
  <PropertyXPathQuery>Params/Param[1]</PropertyXPathQuery>
</ConsolidationProperties>

When consolidation properties are used, the module runs all consolidation calculations in parallel with as many parallel consolidations as there are incoming groups of events. Each calculation will output data that is independent of the other consolidations in progress; subsequent modules in the workflow should take this into account.

TimeXPathQuery

By default, the time value that is used for consolidation evaluation is the time attribute of the incoming data items. It is possible to change this behavior and use a different attribute or element of the incoming data items than the time. It is not recommended that you use this parameter, and you should omit it for most monitoring scenarios.

The TimeControl parameter specifies the window that consolidation occurs in. The exact use of this window is dependent on the CountingCondition parameter that is described later in this document. There are two types of time windows available:

Within time schedule
Generic schedule

Note that the generic schedule cannot be used with the sliding window counting method.

The TimeControl element takes the following parameters:

Parameter	Type	Description
Latency	Integer	The time to wait after the consolidation window for data items that might be delayed. This parameter is optional, and the default is 60 seconds if it is not specified.
GenericSchedule	Complex	Defines a schedule of windows for consolidation to occur. A simple recurring schedule or a more complicated weekly schedule can be defined. Either GenericSchedule or WithinTimeSchedule can be defined, but not both.
WithinTimeSchedule	Complex	Defines a simple time window for consolidation that is triggered by incoming data items. Either GenericSchedule or WithinTimeSchedule can be defined, but not both.

WithinTimeSchedule

This option specifies a time window in seconds. This time window is used to consolidate all data items within the window and output the consolidated data item if the configured conditions are met.

The following code example shows an example configuration of the TimeControl parameter that specifies a window of 60 seconds.

<TimeControl>
  <WithinTimeSchedule>
    <Interval>60</Interval>
  </WithinTimeSchedule>
</TimeControl>

When this type of time control is used, the first data item starts the time window. The time window runs for the specified time. The module can output a consolidated item either during this window, at the end of the window, or not at all, depending on the counting mode configuration that is specified.

Generic Schedule

The generic schedule allows more precise window control and offers a simple recurring schedule or a weekly schedule option. Note that when a generic schedule is being used, the timing window is no longer triggered by incoming data items but is based on a set schedule.

A simple recurring schedule specifies a unit of time and a recurring interval to execute on. If the units are not specified, the value is interpreted as seconds. Optionally, a synchronization time can be specified. The following code example shows a sample generic schedule that uses the simple recurring schedule:

<GenericSchedule>
  <SimpleReccuringSchedule>
    <Interval>120</Interval>
  </SimpleReccuringSchedule>
  <ExcludeDates/>
</GenericSchedule>

To specify a value in units other than seconds, the unit attribute is used. For example:

<GenericSchedule>
  <SimpleReccuringSchedule>
    <Interval Unit="Minutes">30</Interval>
  </SimpleReccuringSchedule>
  <ExcludeDates/>
</GenericSchedule>

The allowed values for the unit attribute are:

Seconds
Minutes
Hours
Days

A synchronization time can be specified as configuration to force the window to align to a specific time of day. The synchronization time is specified in hours and minutes and uses the 24-hour format. For example:

<GenericSchedule>
  <SimpleReccuringSchedule>
    <Interval Unit="Minutes">30</Interval>
    <SyncTime>13:00</SyncTime>
  </SimpleReccuringSchedule>
  <ExcludeDates/>
</GenericSchedule>

The consolidation window does not have to wait until the synchronization time for the first data consolidation. When the schedule is initialized, the next closed time is used, based on the 24-hour schedule. For example, if the preceding configuration was used, and the workflow was initialized at 13:20, the first window starts at 13:30 and then starts every 30 minutes after that point.

When a simple schedule is used, the time window is on a fixed schedule and is not affected by the incoming data items. The point at which data is output for the window is dependent on the counting mode. For the OnNewItemNOP_OnTimerOutputRestart mode, data is output only at the end of each window if at least one data item was received as input during the window. For the other two counting modes, data can be output at any time during the window, based on the number of data items received. The following describes the behavior of each counting mode.

Either zero or one consolidated data item is output during each window for every counting mode. Data items that occur in the window after a consolidated data item has already been output are not processed.

A weekly schedule specifies a more complex set of windows for the consolidator module to evaluate. This can either be a daily schedule or a schedule that spans multiple days.

The daily schedule specifies the days of the week, the start time, and the end time. Multiple windows can be added to the configuration as required. The following example shows a window that runs from Monday to Friday from 9:00 to 10:00:

<GenericSchedule>
  <WeeklySchedule>
    <Windows>
      <Daily>
        <Start>09:00</Start>
        <End>10:00</End>
        <DaysOfWeekMask>62</DaysOfWeekMask>
      </Daily>
    </Windows>
  </WeeklySchedule>
  <ExcludeDates/>
</GenericSchedule>

The DaysOfWeekMask element represents one or more days of the week that the window should run. A single day or multiple days can be specified according to the following values:

Day	Value
Sunday	1
Monday	2
Tuesday	4
Wednesday	8
Thursday	16
Friday	32
Saturday	64

To specify a single day, enter the enumerator value for that day into the DaysOfWeekMask configuration element.

To specify multiple days, add the values for the days together. For example, for Monday, Wednesday and Friday you would specify 42 (2+8+32).

Multiple windows can be specified as shown in the following example, which has two windows, one from 9:00 to 10:00, Monday to Friday and the other 17:00 to 18:00, Monday and Friday:

<GenericSchedule>
  <WeeklySchedule>
    <Windows>
      <Daily>
        <Start>09:00</Start>
        <End>10:00</End>
        <DaysOfWeekMask>62</DaysOfWeekMask>
      </Daily>
<Daily>
        <Start>17:00</Start>
        <End>18:00</End>
        <DaysOfWeekMask>34</DaysOfWeekMask>
      </Daily>
    </Windows>
  </WeeklySchedule>
  <ExcludeDates/>
</GenericSchedule>

The daily schedule does not allow the window to span multiple days. If this is required, you must use the multiple days schedule. This schedule specifies the start and end time, together with a start and end day. The day is specified by using the day values as described previously for the DaysOfWeekMask element.

The following code example shows a window that runs from Monday at 23:00 to Tuesday at 9:00:

<GenericSchedule>
  <WeeklySchedule>
    <Windows>
      <MultipleDays>
        <Start>23:00</Start>
        <StartDayOfWeekMask>2</StartDayOfWeekMask>
        <End>09:00</End>
        <EndDayOfWeekMask>4</EndDayOfWeekMask>
      </MultipleDays>
    </Windows>
  </WeeklySchedule>
  <ExcludeDates />
</GenericSchedule>

Two or more MultipleDays elements can be added, as required. MultipleDays elements can be combined with Daily elements, as required. For example:

<GenericSchedule>
  <WeeklySchedule>
    <Windows>
      <MultipleDays>
        <Start>23:00</Start>
        <StartDayOfWeekMask>2</StartDayOfWeekMask>
        <End>09:00</End>
        <EndDayOfWeekMask>4</EndDayOfWeekMask>
      </MultipleDays>
      <MultipleDays>
        <Start>23:00</Start>
        <StartDayOfWeekMask>8</StartDayOfWeekMask>
        <End>09:00</End>
        <EndDayOfWeekMask>16</EndDayOfWeekMask>
      </MultipleDays>
      <Daily>
        <Start>10:00</Start>
        <End>11:00</End>
        <DaysOfWeekMask>127</DaysOfWeekMask>
      </Daily>
    </Windows>
  </WeeklySchedule>
  <ExcludeDates />
</GenericSchedule>

Specific dates can excluded from the consolidation module when you are using either the simple recurring schedule or the weekly schedule options. Individual days are specified by using the ExcludeDates configuration.

Any number of date ranges can be added, with the constraint that a date range must be contiguous (for example, February 2 to February 4). The year is not specified when you enter the date. You can specify a single date by setting the start and end date to be the same date. The Description can be used optionally to describe why the exclusion has been made, but this does not affect the module behavior in any way.

Dates are entered in the MM/DD format.

The following code example shows two ranges specified. This stops consolidation output for the date range of between February 2 and the February 5, and also on December 1:

<ExcludeDates>
  <DayInterval>
    <Start>02/02</Start>
    <End>02/05</End>
    <Description />
  </DayInterval>
  <DayInterval>
    <Start>12/01</Start>
    <End>12/01</End>
    <Description />
  </DayInterval>
</ExcludeDates>

Latency

The Latency parameter specifies how long to wait after the consolidation window before either outputting data or dropping the current data. The Latency parameter is used for data items that might arrive after the time window due to a delay in delivery. For example, data items that were logged to the local Event log over a network could use the Latency parameter. If these items had arrived on time they would have fallen into the time window and their timestamp would indicate this. Therefore, the consolidator module can wait in case more items are in the window and then retroactively include them in the consolidation. The value of the Latency parameter is specified in seconds and is optional. If it is not specified, the default value is 60 seconds.

The consolidator module does not always wait for the latency period to output data. If sufficient data has arrived to meet the consolidation conditions, the module outputs when it is ready. If the count has not been reached, the module waits to ensure that no more data arrives.

For the OnNewItemNOP_OnTimerOutputRestart counting method, there is no count value specified in the configuration. In this counting mode, the module always waits for the latency before outputting data.

CountingCondition

The CountingCondition configuration defines the method of counting incoming data items. There are three counting methods available. The behavior of each mode is compared through examples in the Remarks section of this document.

The CountingCondition element has two sub-elements:

Parameter	Type	Description
Count	Integer	Specifies how many repeated data items are required within the counting window to trigger output. This element is not required for the OnNewItemNOP_OnTimerOutputRestart counting method.
CountMode	String	Specifies the counting method. There are three values that can be specified in this parameter: OnNewItemNOP_OnTimerOutputRestart OnNewItemTestOutputRestart_OnTimerSlideByOne OnNewItemTestOutputRestart_OnTimerRestart

Count

Integer

Specifies how many repeated data items are required within the counting window to trigger output. This element is not required for the OnNewItemNOP_OnTimerOutputRestart counting method.

CountMode

String

Specifies the counting method. There are three values that can be specified in this parameter:

OnNewItemNOP_OnTimerOutputRestart
OnNewItemTestOutputRestart_OnTimerSlideByOne
OnNewItemTestOutputRestart_OnTimerRestart

OnNewItemNOP_OnTimerOutputRestart

This counting method uses a fixed window that is specified in the TimeControl parameter. A consolidated data item is output only at the end of the window and never during the window. After each window either one or zero data items are output.

When this counting method is specified, no count should be configured for the module; if a count is configured, the module raises an error in the Operations Manager event log and the workflow is not run.

The following code example shows an example configuration for this mode of counting:

<CountingCondition>

<CountMode>OnNewItemNOP_OnTimerOutputRestart</CountMode>
</CountingCondition>

This method performs no evaluation of the count of data items received in the window. If evaluation of the number of data items is required, based on the output of the consolidation, it is necessary to use an expression module after the consolidator module in the workflow.

This method is used for scenarios in which you want to count the number of events over the time window and not stop counting when you reach a threshold. This data can be mapped to an event and stored for reporting purposes, or used to drive the alerting or monitor state.

OnNewItemTestOutputRestart_OnTimerRestart

This counting method uses a fixed window that is specified in the TimeControl parameter and a count value is configured. This count is the threshold for the number of data items that must occur during the timing window that will cause the data items in the window to be consolidated into an output from the consolidator. If this threshold is reached, the module outputs a consolidated data item immediately and does not wait for the end of the window. If the threshold is not reached and the window ends, no data is output from the consolidator. The next data item triggers a new window and disregards the events that fall into the closed window.

The following code example shows an example configuration:

<CountingCondition>
  <Count>3</Count>
  <CountMode>OnNewItemTestOutputRestart_OnTimerRestart</CountMode>
</CountingCondition>

OnNewItemTestOutputRestart_OnTimerSlideByOne

This counting method uses a sliding window that is specified in the TimeControl parameter. It is not possible to use a generic schedule with this method of counting. A simple WithinTimeSchedule configuration must be used. A count value is required for this counting method to specify the threshold for the number of data items that must occur during the timing window. With this method, the window is not fixed after it begins, and the end point can move. This method looks for a set number of data items within the specified window at whatever point the set number occurs.

When the first data item comes to the consolidator, the timing window is started. For subsequent data items, if the count threshold is reached at any point during the window, the consolidator outputs data. If the end of the window is reached, and the threshold has not been reached, the consolidator drops the first data item that came into the window, looks at the time stamp on the second data item, and resets the window to use this as the start time. This process continues until either the count threshold is reached or all data items have occurred more than one window interval in the past.

The following code example shows an example configuration for this module type:

<CountingCondition>
  <Count>5</Count>
  <CountMode>OnNewItemTestOutputRestart_OnTimerSlideByOne</CountMode>
</CountingCondition>

Information


Schema Type	System.ExpressionEvaluatorSchema
Library	System.Library

ConsolidatorType

Schema Definition

Remarks

Information

Additional resources