Azure 串流分析是什麼?What is Azure Stream Analytics?

Azure 串流分析是即時分析與處理複雜事件的引擎,用來同時分析和處理多個來源的大量快速串流資料。Azure Stream Analytics is a real-time analytics and complex event-processing engine that is designed to analyze and process high volumes of fast streaming data from multiple sources simultaneously. 它可以從多個輸入來源 (包括裝置、感應器、點選流、社交媒體摘要和應用程式) 中擷取的資訊,識別模式和關聯性。Patterns and relationships can be identified in information extracted from a number of input sources including devices, sensors, clickstreams, social media feeds, and applications. 這些模式可以用來觸發動作並啟動工作流程,例如建立警示、將資訊提供給報告工具,或是儲存轉換資料以供之後使用。These patterns can be used to trigger actions and initiate workflows such creating alerts, feeding information to a reporting tool, or storing transformed data for later use. 此外,串流分析可用於 Azure IoT Edge 執行階段,並支援與雲端相同的語言或語法。Also, Stream Analytics is available on Azure IoT Edge runtime, and supports the same exact language or syntax as cloud.

下列案例是您何時可以使用 Azure 串流分析的範例:The following scenarios are examples of when you can use Azure Stream Analytics:

  • 從 IoT 裝置分析即時遙測資料流Analyze real-time telemetry streams from IoT devices
  • Web 記錄/點選流分析Web logs/clickstream analytics
  • 車隊管理和自駕車的地理空間分析Geospatial analytics for fleet management and driverless vehicles
  • 高價值資產的遠端監視和預測性維護Remote monitoring and predictive maintenance of high value assets
  • 即時分析銷售點資料以控制庫存和偵測異常Real-time analytics on Point of Sale data for inventory control and anomaly detection

串流分析如何運作?How does Stream Analytics work?

Azure 串流分析作業是由輸入、查詢及輸出所組成。An Azure Stream Analytics job consists of an input, query, and an output. 串流分析內嵌 Azure 事件中樞、Azure IoT 中樞或 Azure Blob 儲存體的資料。Stream Analytics ingests data from Azure Event Hubs, Azure IoT Hub, or Azure Blob Storage. 查詢以 SQL 查詢語言為基礎,可在一段時間內輕鬆地篩選、排序、彙總和聯結串流處理資料。The query, which is based on SQL query language, can be used to easily filter, sort, aggregate, and join streaming data over a period of time. 您也可以使用 JavaScript 和 C# 使用者定義函式 (UDF) 來擴充此 SQL 語言。You can also extend this SQL language with JavaScript and C# user defined functions (UDFs). 透過簡單的語言建構和/或設定執行彙總作業時,您可以輕鬆調整事件排序選項和時間範圍的持續長度。You can easily adjust the event ordering options and duration of time windows when preforming aggregation operations through simple language constructs and/or configurations.

每個作業都具有所轉換資料的輸出,並可控制所要採取的動作以回應您所分析的資訊。Each job has an output for the transformed data, and you can control what happens in response to the information you've analyzed. 例如,您可以:For example, you can:

  • 將資料傳送至服務如 Azure Functions、服務匯流排主題或佇列,以觸發通訊或自訂工作流程下游。Send data to services such as Azure Functions, Service Bus Topics or Queues to trigger communications or custom workflows downstream.
  • 將資料傳送至 Power BI 儀表板以即時顯示在儀表板上。Send data to a Power BI dashboard for real-time dashboarding.
  • 將資料儲存至其他 Azure 儲存體服務,以便根據歷程記錄資料定型機器學習服務模型,或執行批次分析。Store data in other Azure storage services to train a machine learning model based on historical data or perform batch analytics.

下圖顯示如何將資料傳送至串流分析、加以分析並傳送,以進行儲存或呈現等其他動作:The following image shows how data is sent to Stream Analytics, analyzed, and sent for other actions like storage or presentation:

串流分析流程簡介

主要功能和優點Key capabilities and benefits

Azure 串流分析的設計訴求是方便使用、具靈活性、可靠,以及適用於任何作業大小。Azure Stream Analytics is designed to be easy to use, flexible, reliable, and scalable to any job size. 其適用於多個 Azure 區域。It is available across multiple Azure regions. 下圖說明 Azure 串流分析的重要功能:The following image illustrates the key capabilities of Azure Stream Analytics:

串流分析的重要功能

輕鬆開始使用Ease of getting started

您可以輕鬆地開始使用 Azure 串流分析。Azure Stream Analytics is easy to start. 只要按幾下就能連線到多個來源與接收,建立端對端管線。It only takes a few clicks to connect to multiple sources and sinks, creating an end-to-end pipeline. 串流分析可以連線到 Azure 事件中樞Azure IoT 中樞進行串流資料擷取,以及連線到 Azure Blob 儲存體以取得歷史資料。Stream Analytics can connect to Azure Event Hubs and Azure IoT Hub for streaming data ingestion, as well as Azure Blob storage to ingest historical data. 作業輸入也可以包含來自 Azure Blob 儲存體或 SQL 資料庫的靜態或變更緩慢的參考資料,您可以將參考資料加入到串流資料,以執行查閱作業。Job input can also include static or slow-changing reference data from Azure Blob storage or SQL Database that you can join to streaming data to perform lookup operations.

串流分析可將作業輸出路由至許多儲存體系統,例如 Azure Blob 儲存體Azure SQL DatabaseAzure Data Lake StoreAzure CosmosDBStream Analytics can route job output to many storage systems such as Azure Blob storage, Azure SQL Database, Azure Data Lake Store, and Azure CosmosDB. 您可以使用 Azure HDInsight 對儲存的輸出執行批次分析,也可以將輸出傳送至另一個服務 (例如事件中樞) 以供取用,或用於即時呈現視覺效果的 Power BIYou can run batch analytics on stored output with Azure HDInsight, or you can send the output to another service, like Event Hubs for consumption or Power BI for real-time visualization.

如需串流分析輸出的完整清單,請參閱了解來自 Azure 串流分析的輸出For the entire list of Stream Analytics outputs, see Understand outputs from Azure Stream Analytics.

程式設計人員生產力Programmer productivity

Azure 串流分析使用已擴增的簡單 SQL 架構查詢語言,搭配強大的時態性限制式即可分析移動中的資料。Azure Stream Analytics uses a simple SQL-based query language that has been augmented with powerful temporal constraints to analyze data in motion. 若要定義作業轉換,您可以使用簡單的敘述性串流分析查詢語言,來撰寫使用簡單 SQL 建構的複雜時態性查詢和分析。To define job transformations, you use a simple, declarative Stream Analytics query language that lets you author complex temporal queries and analytics using simple SQL constructs. 因為串流分析查詢語言與 SQL 語言一致,只要熟悉 SQL 便可開始建立作業。Because Stream Analytics query language is consistent to the SQL language, familiarity with SQL is sufficient to start creating jobs. 使用 Azure PowerShell、串流分析 Visual Studio 工具串流分析 Visual Studio Code 擴充功能,或 Azure Resource Manager 範本等開發人員工具,也可以建立作業。You can also create jobs by using developer tools like Azure PowerShell, Stream Analytics Visual Studio tools, the Stream Analytics Visual Studio Code extension, or Azure Resource Manager templates. 使用開發人員工具可讓您離線開發轉換查詢,並使用 CI/CD 管線將作業提交至 Azure。Using developer tools allow you to develop transformation queries offline and use the CI/CD pipeline to submit jobs to Azure.

串流分析查詢語言提供了各式各樣的函式,以供分析和處理串流資料。The Stream Analytics query language offers a wide array of functions for analyzing and processing streaming data. 此查詢語言支援簡單的資料操作、彙總函式,和複雜的地理空間函式。This query language supports simple data manipulation, aggregation functions, and complex geospatial functions. 您可以在入口網站中編輯查詢,然後使用從即時資料流所擷取出的範例資料來測試查詢。You can edit queries in the portal and test them using sample data that is extracted from a live stream.

您可以透過定義和叫用其他函式來延伸查詢語言的功能。You can extend the capabilities of the query language by defining and invoking additional functions. 您可以在 Azure Machine Learning 服務定義函式呼叫以利用 Azure Machine Learning 解決方案,並整合 JavaScript 或 C# 使用者定義的函式 (UDF) 或使用者定義的彙總以在串流分析查詢中執行複雜的計算。You can define function calls in the Azure Machine Learning service to take advantage of Azure Machine Learning solutions, and integrate JavaScript or C# user-defined functions (UDFs) or user-defined aggregates to perform complex calculations as part a Stream Analytics query.

完全受控Fully managed

Azure 串流分析是 Azure 上完全受控的無伺服器 (PaaS) 供應項目。Azure Stream Analytics is a fully managed serverless (PaaS) offering on Azure. 您不必佈建任何硬體或管理叢集即可執行您的作業。You don’t have to provision any hardware or manage clusters to run your jobs. Azure 串流分析會全權管理您的作業,它會在雲端設定複雜的計算叢集,並負責處理執行作業所需的效能微調。Azure Stream Analytics fully manages your job by setting up complex compute clusters in the cloud and taking care of the performance tuning necessary to run the job. 與 Azure 事件中樞和 Azure IoT 中樞整合,可讓作業每秒擷取數百萬來自各種來源 (包括連線裝置、點擊資料流及記錄檔等) 的事件。Integration with Azure Event Hubs and Azure IoT Hub allows your job to ingest millions of events per second coming from a number of sources, to include connected devices, clickstreams, and log files. 透過事件中樞的分割功能,您可以將計算分割為邏輯步驟,而每個步驟都可以再進一步地分割來增加延展性。Using the partitioning feature of Event Hubs, you can partition computations into logical steps, each with the ability to be further partitioned to increase scalability.

在雲端或在智慧邊緣執行Run in the cloud or on the intelligent edge

Azure 串流分析可以在雲端執行以進行大規模分析,或在 IoT Edge 上執行以進行超低延遲分析。Azure Stream Analytics can run in the cloud, for large-scale analytics, or run on IoT Edge for ultra-low latency analytics. Azure 串流分析會在雲端和智慧邊緣上使用相同的查詢語言,讓開發人員能夠建置真正的混合式架構進行串流處理。Azure Stream Analytics uses the same query language on both cloud and the edge, enabling developers to build truly hybrid architectures for stream processing.

低擁有權總成本Low total cost of ownership

作為雲端服務,串流分析已進行成本最佳化。As a cloud service, Stream Analytics is optimized for cost. 不會有預付費用 - 您只需針對您耗用的串流單位和處理的資料量付費。There are no upfront costs involved - you only pay for the streaming units you consume, and the amount of data processed. 不需要承諾或佈建叢集,您可以根據業務需求相應增加或減少作業。There is no commitment or cluster provisioning required, and you can scale the job up or down based on your business needs.

任務關鍵性就緒Mission-critical ready

Azure 串流分析適用於全球多個區域,其設計訴求是藉由支援可靠性、安全性和合規性需求來執行任務關鍵性工作負載。Azure Stream Analytics is available across multiple regions worldwide and is designed to run mission-critical workloads by supporting reliability, security and compliance requirements.

可靠性Reliability

Azure 串流分析可保證僅只一次的事件處理,以及至少一次的事件傳遞,因此永遠不會遺失事件。Azure Stream Analytics guarantees exactly-once event processing and at-least-once delivery of events, so events are never lost. 事件傳遞保證所述,保證選取的輸出恰好處理一次。Exactly-once processing is guaranteed with selected output as described in Event Delivery Guarantees.

Azure 串流分析具有可在事件傳遞失敗時進行復原的內建功能。Azure Stream Analytics has built-in recovery capabilities in case the delivery of an event fails. 串流分析也會提供內建檢查點來維護作業的狀態,並提供可重複出現的結果。Stream Analytics also provides built-in checkpoints to maintain the state of your job and provides repeatable results.

串流分析為受控服務,可保證事件處理在分鐘層級細微性上有 99.9% 的可用性。As a managed service, Stream Analytics guarantees event processing with a 99.9% availability at a minute level of granularity. 如需詳細資訊,請參閱串流分析 SLA 頁面。For more information, see the Stream Analytics SLA page.

安全性Security

就安全性而言,Azure 串流分析會將所有傳入和傳出的通訊加密,並支援 TLS 1.2。In terms of security, Azure Stream Analytics encrypts all incoming and outgoing communications and supports TLS 1.2. 內建的檢查點也會加密。Built-in checkpoints are also encrypted. 串流分析不會儲存傳入資料,因為所有處理都在記憶體內進行。Stream Analytics doesn't store the incoming data since all processing is done in-memory.

法規遵循Compliance

Azure 串流分析會遵循如 Azure 合規性概觀中所述的多個合規性認證。Azure Stream Analytics follows multiple compliance certifications as described in the overview of Azure compliance.

效能Performance

串流分析每秒可以處理數百萬個事件,並可極低延遲地傳遞結果。Stream Analytics can process millions of events every second and it can deliver results with ultra low latencies. 它可讓您相應增加及向外延展,以處理大型的即時和複雜事件處理應用程式。It allows you to scale-up and scale-out to handle large real-time and complex event processing applications. 串流分析藉由下列方式來支援更高的效能:進行分割,並允許在多個串流節點上平行化及執行複雜的查詢。Stream Analytics supports higher performance by partitioning, allowing complex queries to be parallelized and executed on multiple streaming nodes. Azure 串流分析是以 Trill 為基礎,這是與 Microsoft Research 合作開發的高效能記憶體內部串流分析引擎。Azure Stream Analytics is built on Trill, a high-performance in-memory streaming analytics engine developed in collaboration with Microsoft Research.

後續步驟Next steps

您現在已大致了解 Azure 串流分析。You now have an overview of Azure Stream Analytics. 接下來,您可以深入了解並建立您的第一個串流分析作業:Next, you can dive deep and create your first Stream Analytics job: