Stream data from Social Engagement to Microsoft Azure Event Hubs

Start building out your custom metrics or analysis based on the posts found by Microsoft Social Engagement. Using Microsoft Azure Event Hubs—a highly scalable publish-subscribe event ingestor that can intake millions of events per second—you can transform and store social data for further processing. Specify an automation rule in Social Engagement and stream all posts that match a data set as events to a connected event hub. Once you have the data ingested into the event hub, you can access the data from any application that you connected to your event hub.

In Social Engagement, go to Settings > Connections to connect your Social Engagement solution to Azure Event Hubs.

Prerequisites to stream posts to Azure Event Hubs

To stream posts to Azure Event Hubs, the following prerequisites must be met:

  • You have access to an existing event hub, and you also have its connection string with Send rights. For more information about how to create an Azure Event Hubs and how to get the connection string, see Microsoft Azure: Get started with Event Hubs.

  • You have a Social Engagement Enterprise license assigned to your Microsoft Office 365 user account. Although Administrators with Professional licenses can configure the connection to an event hub, you’ll need an Enterprise license to create automation rules that stream data to a connected event hub. More information: Integrate Social Engagement with Office 365

  • You have an Administrator role in Social Engagement. More information: Understand user roles

Connect Social Engagement to Azure Event Hubs

Social Engagement allows you to connect to multiple event hubs. Before you can start to stream posts to Azure Event Hubs, you need to enable the connection and acknowledge that you agree data can be sent to a third-party application. More information: Privacy Notice

Connect to Azure Event Hubs

  1. In Social Engagement, go to Settings > Connections.

  2. In the Connections pane, select Microsoft Azure Event Hubs from the list.

  3. When adding your first Azure Event Hubs connection, you will be shown a disclaimer. Read the disclaimer carefully, and then click Continue to proceed setting up the connection. More information: Privacy Notice

  4. In the Connect to Azure Event Hubs pane, enter an Event hub name and the Connection string for the event hub you want to connect.

    For more information about where to find the connection string, see Microsoft Azure: Get started with Event Hubs.

  5. Click Save Save button.

Social Engagement as a data source in Customer Insights

Dynamics 365 Customer Insights includes a connector to add Social Engagement as a data source. This connector creates and manages the connection between Social Engagement and Azure Event Hubs. For more information about the connector, see Add Microsoft Social Engagement as a data source

Disable or re-enable a connection to Azure Event Hubs

Disable a connection if you want to pause the stream of posts to the connected event hubs. Re-enable the connection to start the stream again. Posts that were acquired while the connection to an event hub was disabled will not be processed retroactively.

Important

Before you disable a connection, make sure no active automation rule is configured to stream posts to an event hub. We’ll inform you with an error message if you try to disable a connection that is used in active automation rules. More information: Route posts using automation rules

Disable a connection

  1. In Social Engagement, go to Settings > Connections.

  2. In the Connections pane, select Microsoft Azure Event Hubs from the list.

  3. Select the connection you want to disable and set Enable connection to Off.

  4. Click Save Save button.

Re-enable a disabled connection

  1. In Social Engagement, go to Settings > Connections.

  2. In the Connections pane, select Microsoft Azure Event Hubs from the list.

  3. Select the connection you want to re-enable and set Enable connection to On.

  4. Click Save Save button.

Choose the data to stream to event hubs using automation rules

Once you established the connection to Azure Event Hubs, you need to select which data is passed on to your event hub. To do this, go to Settings > Automation Rules, and then create a new automation rule. Select the search topics and filters you want to apply to your data set. Then, select the Stream to Event Hubs check box. In the Azure Event Hubs drop-down list, select the event hub to stream social posts to. For detailed information and procedures about how to set up automation rules, see Route posts using automation rules.

Automation rule settings to stream social posts to Azure Event Hubs

Tip

If you find a data set that you want to use in an automation rule while working with the application, click the More options with the current filters button More options with current filters. This populates the filters of the automation rule with the currently-active filters for you. For more information about the areas of Social Engagement that provide this feature, see Explore more options with your data set.

When Social Engagement sends a social post as an event to an Azure Event Hubs, a JSON payload is sent to the connected event hub. Along with the default properties of the post, you can define custom key-value pairs to add when you set up the automation rule for a given data set. These additional properties will show in the metadata section of the JSON payload. More information: JSON reference for events from Social Engagement

To work with the events that Social Engagement streams to an event hub, you have several options to define the processing and use the data. For an example about how to create a Microsoft Power BI dashboard by using Azure Stream Analytics queries, see Work with events from Social Engagement in Azure Event Hubs, Step 4: Create an Azure Stream Analytics job.

Privacy notice

By enabling Social Engagement to connect to Azure Event Hubs, you will allow social data to be streamed to event hubs using automation rules. Azure Event Hubs stores the social data streamed from Social Engagement for a pre-configured period of time and any application that can listen to the event hub will be able to access, store and/or process this data.
Note that the social data sent from Social Engagement includes information regarding the social post (author and text) as well as enriched information such as Language, Location, Sentiment, Tags, etc. For complete information on the content of a social post sent to event hubs, please see the JSON schema definition.

See Also

Manage connections in Social Engagement
Work with events from Social Engagement in Azure Event Hubs
Administer Microsoft Social Engagement
JSON reference for events from Social Engagement