Using Event Sources and Event Sinks (StreamInsight)

  • Article

 

The StreamInsight server consists of event sources, a set of standing processes that use queries to examine streams of events, and event sinks which process the results. StreamInsight supports event sources and sinks that implement the IObservable and IEnumerable interfaces, and the IQbservable, IQueryable, and IQStreamable interfaces.

Using Observable and Enumerable Event Sources and Event Sinks

StreamInsight supports event sources and event sinks that implement the IObservable and IEnumerable interfaces. These interfaces are the dominant general-purpose contracts in the .NET Framework for pulling (IEnumerable) and pushing (IObservable) data among the components of an application.

While the IEnumerable and IEnumerator interfaces are already familiar to .NET developers, the IObservable and IObserver interfaces were introduced more recently in the .NET Framework 4. The IObservable development model is based on the observable/observer design pattern. In this design pattern:

  • An observer is any object that wishes to be notified when the state of another object changes.

  • An observable is a push-based sequence, in which another object may register an interest.

In a StreamInsight application, the observable is the event source. The query acts as an observer of this source, and presents its result as an observable to the event sink, which is in turn an observer of the query as the source. For more information, see Exploring the Observer Design Pattern on MSDN.

The tasks for the application developer to enable this development model are minimal:

  1. Provide a class that implements the IObservable or IEnumerable interface as the event producer.

  2. Provide a class that implements the IObserver interface, or a class that consumes an IEnumerable, as the event consumer.

  3. Bind these objects to queries.

This development model does not require a major learning or programming investment to define the temporal properties of the events, or awareness of the underlying state transitions of the core adapter model. Here are some scenarios in which this development model is appropriate:

  1. Historical queries. Financial analyst Alice identifies patterns in historical market data by executing temporal queries over large data sets. In this case, the input stream has a definite end – after the last data element is processed, the query should end.

  2. Ad hoc queries over live sources. Server administrator Bob suspects a denial of service attack. He identifies the source of the attack through an iterative series of ad hoc queries over the streams of network data that he is monitoring.

  3. Embedded in custom applications. User interface developer Charlie analyzes patterns of mouse gestures by using temporal queries over Windows events. StreamInsight is embedded in the custom application.

In each of these cases, the query exists only so long as the client application cares to listen to the results.

Programming considerations for observable and enumerable sources and sinks

Observable and enumerable sources and sinks are supported in embedded and server solutions

Observable or enumerable event sources and sinks are supported in the hosted deployment model, where the StreamInsight server is embedded in the application, and in the standalone or remote server deployment model when you deploy the query and sink to the server. For more information on deployment models, see StreamInsight Server Deployment Models.

API features support the temporal requirements of StreamInsight

The API permits full fidelity with StreamInsight’s temporal model, including the enqueuing of CTI events.

  • On observable and enumerable inputs:

    • The To[Point|Interval|Edge]Stream methods wrap the input data as StreamInsight events and specify the temporal characteristics of the source data.

    • A helper method and properties on the AdvanceTimeSettings class help the developer to specify the frequency at which CTI events are enqueued, thereby determining the liveliness of the input data.

  • On observable and enumerable outputs, the developer can choose whether or not to incorporate time information in the form of point, interval or edge events through the To[Point|Interval|Edge]Observable and To[Point|Interval|Edge]Enumerable methods.

Using IQbservable, IQueryable, and IQStreamable Event Sources and Event Sinks

There are various LINQ providers in existence today: LINQ to Objects, LINQ to SQL, and the Reactive Extensions for .NET. StreamInsight has its own LINQ provider which enables the developer to express temporal queries over the CepStreamtype.

To support the composition of LINQ queries over sequences and temporal streams, StreamInsight exposes implementations of the IQueryable, IQbservable, and IQStreamable specializations of source types. These interfaces are defined in the System.Linq, System.Reactive.Linq, and Microsoft.ComplexEventProcessing.Linq namespaces.

A user can switch across the source types (IQueryable, IQbservable, and IQStreamable) by using bridge methods. A bridge method begins with the prefix "To". For example, a user can bridge an IQStreamable type into an IQbservable type by using the ToObservable() method. The bridge methods supported in StreamInsight are shown in Figure 1. Their signatures (in extension) can be found in API Support for IQbservable, IQueryable, IQStreamable in StreamInsight.

Bridge methods between source types
Figure 1: Bridge methods to switch across source types