Configure your Event Hubs

Applies to:

Learn how to configure your Event Hubs so that it can ingest events from Microsoft 365 Defender.

Set up the required Resource Provider in the Event Hubs subscription

  1. Sign in to the Azure portal.
  2. Select Subscriptions > { Select the subscription the event hubs will be deployed to } > Resource providers.
  3. Verify whether the Microsoft.Insights Provider is registered. Otherwise, register it.

The list of service providers page in the Microsoft Azure portal

Set up Azure Active Directory App Registration

Note

You must have Administrator role or Azure Active Directory (AAD) must be set to allow non-Administrators to register apps. You must also have an Owner or User Access Administrator role to assign the service principal a role. For more information, see Create an Azure AD app & service principal in the portal - Microsoft identity platform | Microsoft Docs.

  1. Create a new registration (which inherently creates a service principal) in Azure Active Directory > App registrations > New registration.

  2. Fill out the form with just the Name (no Redirect URI is required).

    The application name display section in the Microsoft Azure portal

    The Overview information section in the Microsoft Azure portal

  3. Create a secret by clicking on Certificates & secrets > New client secret:

    The Client secret section in the Microsoft Azure portal

This client secret value is used by Microsoft Graph APIs to authenticate this application that is being registered.

Warning

You won't be able to access the client secret again so make sure to save it.

Set up Event Hubs namespace

  1. Create an Event Hubs Namespace:

    Go to Event Hub > Add and select the pricing tier, throughput units and Auto-Inflate (requires standard pricing and under features) appropriate for the load you're expecting. For more information, see Pricing - Event Hubs | Microsoft Azure.

    Note

    You can use an existing event-hub, but the throughput and scaling are set at the namespace level so it is recommended to place an event-hub in its own namespace.

    The event hubs section in the Microsoft Azure portal

  2. You'll also need the Resource ID of this Event Hubs Namespace. Go to your Azure Event Hubs namespace page > Properties. Copy the text under Resource ID and record it for use during the Microsoft 365 Configuration section below.

    The event hubs properties section in the Microsoft Azure portal

Add permissions

You're required to add permissions to the following roles to entities that are involved in Event Hubs data management:

  • Contributor: The permissions related to this role are added to entity who logs in to the Microsoft 365 Defender portal.
  • Reader and Azure Event Hub data Receiver: The permissions related to these roles are assigned to the entity who is already assigned the role of a Service Principal and logs in to the Azure Active Directory application.

To ensure that these roles have been added, perform the following step:

Go to Event Hub Namespace > Access Control (IAM) > Add and verify under Role assignments.

An application registration service principal section in the Microsoft Azure portal

Set up Event Hubs

Option 1:

You can create an Event Hubs within your Namespace and all the Event Types (Tables) you select to export will be written into this one Event Hub.

Option 2:

Instead of exporting all the Event Types (Tables) into one Event Hub, you can export each table into different Event Hubs inside your Event Hubs Namespace (one Event Hub per Event Type).

In this option, Microsoft 365 Defender will create Event Hubs for you.

Note

If you are using an Event Hub Namespace that is not part of an Event Hub Cluster, you will only be able to choose up to 10 Event Types (Tables) to export in each Export Settings you define, due to an Azure limitation of 10 Event Hub per Event Hub Namespace.

For example:

An event hubs section in the Microsoft Azure portal

If you choose this option, you can skip to the Configure Microsoft 365 Defender to send email tables section.

Create Event Hubs within your Namespace by selecting Event Hub > + Event Hub.

The Partition Count allows for more throughput via parallelism, so it's recommended to increase this number based on the load you're expecting. Default Message Retention and Capture values of 1 and Off are recommended.

An event hubs creation section in the Microsoft Azure portal

For these Event Hubs (not namespace), you'll need to configure a Shared Access Policy with Send, Listen Claims. Click on your Event Hub > Shared access policies > + Add and then give it a Policy name (not used elsewhere) and check Send and Listen.

The Shared access policies page in the Microsoft Azure portal

Configure Microsoft 365 Defender to send email tables

Set up Microsoft 365 Defender send Email tables to Splunk via Event Hubs

  1. Sign in to Microsoft 365 Defender with an account that meets all the following role requirements:

    • Contributor role at the Event Hubs Namespace Resource level or higher for the Event Hubs that you'll be exporting to. Without this permission, you'll get an export error when you try to save the settings.

    • Global Admin or Security Admin Role on the tenant tied to Microsoft 365 Defender and Azure.

      The Settings page of the Microsoft 365 Defender portal

  2. Click on Raw Data Export > +Add.

    You'll now use the data that you recorded above.

    Name: This value is local and should be whatever works in your environment.

    Forward events to event hub: Select this checkbox.

    Event-Hub Resource ID: This value is the Event Hubs Namespace Resource ID you recorded when you set up the Event Hubs.

    Event-Hub name: If you created an Event Hubs inside your Event Hubs Namespace, paste the Event Hubs name you recorded above.

    If you choose to let Microsoft 365 Defender to create Event Hubs per Event Types (Tables) for you, leave this field empty.

    Event Types: Select the Advanced Hunting tables that you want to forward to the Event Hubs and then on to your custom app. Alert tables are from Microsoft 365 Defender, Devices tables are from Microsoft Defender for Endpoint (EDR), and Email tables are from Microsoft Defender for Office 365. Email Events records all Email Transactions. The URL (Safe Links), Attachment (Safe Attachments), and Post Delivery Events (ZAP) are also recorded and can be joined to the Email Events on the NetworkMessageId field.

    The Streaming API settings page in the Microsoft Azure portal

  3. Make sure to click Submit.

Verify that the events are being exported to the Event Hubs

You can verify that events are being sent to the Event Hubs by running a basic Advanced Hunting query. Select Hunting > Advanced Hunting > Query and enter the following query:

EmailEvents
|join kind=fullouter EmailAttachmentInfo on NetworkMessageId
|join kind=fullouter EmailUrlInfo on NetworkMessageId
|join kind=fullouter EmailPostDeliveryEvents on NetworkMessageId
|where Timestamp > ago(1h)
|count

This query will show you how many emails were received in the last hour joined across all the other tables. It will also show you if you're seeing events that could be exported to the event hubs. If this count shows 0, then you won't see any data going out to the Event Hubs.

The advanced hunting page in the Microsoft Azure portal

Once you've verified there's data to export, you can view the Event Hubs page to verify that messages are incoming. This process can take up to one hour.

  1. In Azure, go to Event Hub > Click on the Namespace > Event Hub > Click on the Event Hub.
  2. Under Overview, scroll down and in the Messages graph you should see Incoming Messages. If you don't see any results, then there will be no messages for your custom app to ingest.

 The Overview page in the Microsoft 365 Azure portal