Azure 事件中樞 — 巨量資料串流平台和事件擷取服務Azure Event Hubs — A big data streaming platform and event ingestion service

Azure 事件中樞是巨量資料串流平台和事件擷取服務。Azure Event Hubs is a big data streaming platform and event ingestion service. 其每秒可接收和處理數百萬個事件。It can receive and process millions of events per second. 傳送至事件中樞的資料可以透過任何即時分析提供者或批次/儲存體配接器來轉換和儲存。Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters.

下面是一些可以使用事件中樞的案例:The following scenarios are some of the scenarios where you can use Event Hubs:

  • 異常偵測 (詐騙/極端值)Anomaly detection (fraud/outliers)
  • 應用程式記錄檔Application logging
  • 分析管線 (例如點選流)Analytics pipelines, such as clickstreams
  • 即時儀表板Live dashboarding
  • 封存資料Archiving data
  • 交易處理Transaction processing
  • 使用者遙測處理User telemetry processing
  • 裝置遙測串流Device telemetry streaming

為何使用事件中樞?Why use Event Hubs?

只有在能輕鬆地處理資料並從資料來源中獲得即時見解時,資料才顯得有價值。Data is valuable only when there is an easy way to process and get timely insights from data sources. 事件中樞提供低延遲的分散式串流處理平台,並與 Azure 內部或外部的資料及分析服務完美整合,藉以建置完整的巨量資料管線。Event Hubs provides a distributed stream processing platform with low latency and seamless integration, with data and analytics services inside and outside Azure to build your complete big data pipeline.

事件中樞代表事件管線的「大門」,在方案架構中通常稱為「事件擷取器」 。Event Hubs represents the "front door" for an event pipeline, often called an event ingestor in solution architectures. 事件擷取器是介於事件發佈者和事件取用者之間的元件或服務,它能將事件串流的生產與這些事件的取用彼此脫鉤。An event ingestor is a component or service that sits between event publishers and event consumers to decouple the production of an event stream from the consumption of those events. 事件中樞提供具有時間保留緩衝的統一串流平台,可分開處理事件產生者和事件取用者。Event Hubs provides a unified streaming platform with time retention buffer, decoupling event producers from event consumers.

下列各節會說明 Azure 事件中樞服務的主要功能:The following sections describe key features of the Azure Event Hubs service:

完全受控 PaaSFully managed PaaS

事件中樞是僅需一點設定或管理成本的完全受控平台即服務 (PaaS),因此您可將重心放在商務解決方案上。Event Hubs is a fully managed Platform-as-a-Service (PaaS) with little configuration or management overhead, so you focus on your business solutions. Apache Kafka 生態系統的事件中樞提供您無須管理、設定或執行叢集的 PaaS Kafka 體驗。Event Hubs for Apache Kafka ecosystems gives you the PaaS Kafka experience without having to manage, configure, or run your clusters.

支援即時和批次處理Support for real-time and batch processing

即時內嵌、緩衝、儲存及處理您的資料流,以取得可採取動作的見解。Ingest, buffer, store, and process your stream in real time to get actionable insights. 事件中樞會使用分割的取用者模型,可讓多個應用程式同時處理資料流,並讓您控制處理的速度。Event Hubs uses a partitioned consumer model, enabling multiple applications to process the stream concurrently and letting you control the speed of processing.

近乎即時地擷取 Azure Blob 儲存體Azure Data Lake Storage  中的資料,以用於長期保留或微批次處理。Capture your data in near-real time in an Azure Blob storage or Azure Data Lake Storage for long-term retention or micro-batch processing. 您可以在用來取得即時分析的相同資料流上實現此行為。You can achieve this behavior on the same stream you use for deriving real-time analytics. 擷取事件資料的作業很快就能設定完成。Setting up capture of event data is fast. 執行作業時不需要系統管理成本,而且它可以針對事件中樞的 輸送量單位自動進行調整。There are no administrative costs to run it, and it scales automatically with Event Hubs throughput units. 事件中樞可讓您專注於資料處理,而不是資料擷取。Event Hubs enables you to focus on data processing rather than on data capture.

Azure 事件中樞也整合了 Azure Functions 來達到無伺服器架構。Azure Event Hubs also integrates with Azure Functions for a serverless architecture.

可調整Scalable

透過事件中樞,您可以先使用以 MB 為單位的資料流,然後成長到 GB 或 TB。With Event Hubs, you can start with data streams in megabytes, and grow to gigabytes or terabytes. 自動擴充功能是用來調整輸送量單位數目以符合使用量需求的多個選項之一。The Auto-inflate feature is one of the many options available to scale the number of throughput units to meet your usage needs.

豐富的生態系統Rich ecosystem

透過以業界標準 AMQP 1.0 通訊協定為基礎且適用於各種語言 .NETJavaPythonJavaScript 的廣大生態系統,您可以輕鬆地開始處理來自事件中樞的串流。With a broad ecosystem based on the industry-standard AMQP 1.0 protocol and available in various languages .NET, Java, Python, JavaScript, you can easily start processing your streams from Event Hubs. 所有支援的用戶端語言皆提供低階整合。All supported client languages provide low-level integration. 生態系統也可讓您緊密整合 Azure 串流分析和 Azure Functions 等 Azure 服務,進而讓您建置無伺服器的架構。The ecosystem also provides you with seamless integration with Azure services like Azure Stream Analytics and Azure Functions and thus enables you to build serverless architectures.

此外,Apache Kafka 生態系統的事件中樞可讓 Apache Kafka (1.0 版和更新版本) 用戶端和應用程式與事件中樞通訊。Event Hubs for Apache Kafka ecosystems furthermore enables Apache Kafka (1.0 and later) clients and applications to talk to Event Hubs. 您不需要安裝、設定及管理自己的 Kafka 和 Zookeeper 叢集,或使用非 Azure 原生的一些 Kafka 即服務供應項目。You do not need to set up, configure, and manage your own Kafka and Zookeeper clusters or use some Kafka-as-a-Service offering not native to Azure.

重要架構元件Key architecture components

事件中樞包含下列重要元件Event Hubs contains the following key components:

  • 事件產生者:將資料傳送至事件中樞的任何實體。Event producers: Any entity that sends data to an event hub. 事件發佈者可以使用 HTTPS、AMQP 1.0 或 Apache Kafka (1.0 版或更新版本) 發佈事件Event publishers can publish events using HTTPS or AMQP 1.0 or Apache Kafka (1.0 and above)
  • 分割區:每一個取用者只會讀取訊息資料流的特定子集或分割區。Partitions: Each consumer only reads a specific subset, or partition, of the message stream.
  • 取用者群組:檢視整個事件中樞 (狀態、位置或位移) 的窗口。Consumer groups: A view (state, position, or offset) of an entire event hub. 取用者群組能讓取用應用程式各自擁有獨立的事件串流檢視。Consumer groups enable consuming applications to each have a separate view of the event stream. 取用應用程式會依自己的步調以及自己的位移獨立讀取串流。They read the stream independently at their own pace and with their own offsets.
  • 輸送量單位:預先購買的容量單位,可控制事件中樞的輸送量容量。Throughput units: Pre-purchased units of capacity that control the throughput capacity of Event Hubs.
  • 事件接收者:從事件中樞讀取事件資料的任何實體。Event receivers: Any entity that reads event data from an event hub. 所有事件中樞取用者均透過 AMQP 1.0 工作階段來連線。All Event Hubs consumers connect via the AMQP 1.0 session. 事件中樞服務會透過工作階段傳遞可用的事件。The Event Hubs service delivers events through a session as they become available. 所有 Kafka 取用者都是透過 Kafka 通訊協定 1.0 和更新版本連線。All Kafka consumers connect via the Kafka protocol 1.0 and later.

下圖顯示事件中樞串流處理架構︰The following figure shows the Event Hubs stream processing architecture:

事件中樞

Azure Stack Hub 上的事件中樞Event Hubs on Azure Stack Hub

Azure Stack Hub 上的事件中樞可讓您實現混合式雲端情況。Event Hubs on Azure Stack Hub allows you to realize hybrid cloud scenarios. 針對內部部署和 Azure 雲端處理,支援串流和事件型解決方案。Streaming and event-based solutions are supported, for both on-premises and Azure cloud processing. 無論是混合式 (已連線) 或已中斷連線,您的解決方案都可以支援大規模的事件/資料流處理。Whether your scenario is hybrid (connected), or disconnected, your solution can support processing of events/streams at large scale. 您的情況只會受到事件中樞叢集大小的限制,此限制可以根據自己的需求佈建。Your scenario is only bound by the Event Hubs cluster size, which you can provision according to your needs.

事件中樞版本 (位於 Azure Stack Hub 和 Azure 上) 提供高度功能同位。The Event Hubs editions (on Azure Stack Hub and on Azure) offer a high degree of feature parity. 此同位表示 SDK、範例、PowerShell、CLI 和入口網站提供類似的體驗,但有一些差異。This parity means SDKs, samples, PowerShell, CLI, and portals offer a similar experience, with few differences.

在公開預覽期間,堆疊上的事件中樞可免費使用。Event Hubs on Stack is free during public preview. 如需詳細資訊,請參閱 Azure Stack Hub 上的事件中樞概觀For more information, see Event Hubs on Azure Stack Hub overview.

後續步驟Next steps

若要開始使用事件中樞,請參閱 傳送及接收事件 教學課程:To get started using Event Hubs, see the Send and receive events tutorials:

若要深入了解事件中樞,請參閱下列文章:To learn more about Event Hubs, see the following articles: