Enable capturing of events streaming through Azure Event Hubs
You can configure Capture at the event hub creation time using the Azure portal. You can either capture the data to an Azure Blob storage container, or to an Azure Data Lake Storage Gen 1 or Gen 2 account.
For more information, see the Event Hubs Capture overview.
The destination storage (Azure Storage or Azure Data Lake Storage) account must be in the same subscription as the event hub.
Capture data to Azure Storage
When you create an event hub, you can enable Capture by clicking the On button in the Create Event Hub portal screen. You then specify a Storage Account and container by clicking Azure Storage in the Capture Provider box. Because Event Hubs Capture uses service-to-service authentication with storage, you do not need to specify a storage connection string. The resource picker selects the resource URI for your storage account automatically. If you use Azure Resource Manager, you must supply this URI explicitly as a string.
The default time window is 5 minutes. The minimum value is 1, the maximum 15. The Size window has a range of 10-500 MB.
You can enable or disable emitting empty files when no events occur during the Capture window.
Capture data to Azure Data Lake Storage Gen 2
Follow Create a storage account article to create an Azure Storage account. Set Hierarchical namespace to Enabled on the Advanced tab to make it an Azure Data Lake Storage Gen 2 account.
When creating an event hub, do the following steps:
Select On for Capture.
Select Azure Storage as the capture provider. The Azure Data Lake Store option you see for the Capture provider is for the Gen 1 of Azure Data Lake Storage. To use a Gen 2 of Azure Data Lake Storage, you select Azure Storage.
Select the Select Container button.
Select the Azure Data Lake Storage Gen 2 account from the list.
Select the container (file system in Data Lake Storage Gen 2).
On the Create Event Hub page, select Create.
The container you create in a Azure Data Lake Storage Gen 2 using this user interface (UI) is shown under File systems in Storage Explorer. Similarly, the file system you create in a Data Lake Storage Gen 2 account shows up as a container in this UI.
Capture data to Azure Data Lake Storage Gen 1
To capture data to Azure Data Lake Storage Gen 1, you create a Data Lake Storage Gen 1 account, and an event hub:
Create an Azure Data Lake Storage Gen 1 account and folders
- Create a Data Lake Storage account, following the instructions in Get started with Azure Data Lake Storage Gen 1 using the Azure portal.
- Follow the instructions in the Assign permissions to Event Hubs section to create a folder within the Data Lake Storage Gen 1 account in which you want to capture the data from Event Hubs, and assign permissions to Event Hubs so that it can write data into your Data Lake Storage Gen 1 account.
Create an event hub
The event hub must be in the same Azure subscription as the Azure Data Lake Storage Gen 1 account you created. Create the event hub, clicking the On button under Capture in the Create Event Hub portal page.
In the Create Event Hub portal page, select Azure Data Lake Store from the Capture Provider box.
In Select Store next to the Data Lake Store drop-down list, specify the Data Lake Storage Gen 1 account you created previously, and in the Data Lake Path field, enter the path to the data folder you created.
Add or configure Capture on an existing event hub
You can configure Capture on existing event hubs that are in Event Hubs namespaces. To enable Capture on an existing event hub, or to change your Capture settings, click the namespace to load the overview screen, then click the event hub for which you want to enable or change the Capture setting. Finally, click the Capture option on the left side of the open page and then edit the settings, as shown in the following figures:
Azure Blob Storage
Azure Data Lake Storage Gen 2
Azure Data Lake Storage Gen 1
- Learn more about Event Hubs capture by reading the Event Hubs Capture overview.
- You can also configure Event Hubs Capture using Azure Resource Manager templates. For more information, see Enable Capture using an Azure Resource Manager template.
- Learn how to create an Azure Event Grid subscription with an Event Hubs namespace as its source
- Get started with Azure Data Lake Store using the Azure portal