question

LangfordManyau-8202 avatar image
0 Votes"
LangfordManyau-8202 asked PRADEEPCHEEKATLA-MSFT commented

Azure Eventhunt to Azure data lake

I want to send data from eventhub to data lake. Is this possible?

azure-data-lake-storageazure-event-hubs
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hello @LangfordManyau-8202,

Welcome to the Microsoft Q&A platform.

Yes, it is possible to send data from eventhub to data lake.

Event Hubs Capture enables you to automatically capture the streaming data in Event Hubs and save it to your choice of either a Blob storage account, or an Azure Data Lake Storage account. You can enable capture from the Azure portal, and specify a minimum size and time window to perform the capture. Using Event Hubs Capture, you specify your own Azure Blob Storage account and container, or Azure Data Lake Storage account, one of which is used to store the captured data. Captured data is written in the Apache Avro format.

Steps to enable capture on Azure Blob Storage account/Azure Data Lake Gen2.

Note: While selecting the container you can choose Azure Blob Storage or ADLS Gen2 account and select the container.

130544-image.png

Steps to enable capture on Azure Data Lake account:

130633-image.png

For more details, refer to Enable capturing of events streaming through Azure Event Hubs.

Hope this will help. Please let us know if any further queries. Thank you.


  • Please don't forget to click on 130596-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators


image.png (114.3 KiB)
image.png (132.1 KiB)
image.png (2.6 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you very much, i managed to captured the event data as avro file format, is there a way to change file format so that i can receive data as parquet format

0 Votes 0 ·

Hello @LangfordManyau-8202,

Captured data is written in Apache Avro format.

No, you cannot change the file format.

Reason: Azure Event Hub service doesn't know about the payload format it is processing and it stores the events in parquet format.

Hope this will help. Please let us know if any further queries.

0 Votes 0 ·
Taz-3478 avatar image
1 Vote"
Taz-3478 answered Taz-3478 edited

Hello @LangfordManyau-8202 !


Hope you are having a great day!

You can refer to the documentation of "Use Azure Data Lake Storage Gen1 to capture data from Event Hubs" for which i have attached the link below.
https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-archive-eventhub-capture.


Let me know if you need more assistance. I will be glad to help you!

Thank you,
Tasadduq



| Please don't forget to Upvote and Accept as answer if the reply is helpful |

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.