什麼是串流分析?What is Stream Analytics?

Azure 串流分析是事件處理引擎,可讓您檢查來自裝置的大量資料流。Azure Stream Analytics is an event-processing engine that allows you to examine high volumes of data streaming from devices. 傳入的資料可能來自裝置、感應器、網站、社交媒體摘要和應用程式等。Incoming data can be from devices, sensors, web sites, social media feeds, applications, and more. 它也支援從資料流擷取資訊,識別模式和關聯性。It also supports extracting information from data streams, identifying patterns, and relationships. 然後,您可以使用這些模式來觸發其他動作下游 (例如警示)、將資訊提供給報告工具,或是儲存資訊以供之後使用。You can then use these patterns to trigger other actions downstream, like alerts, feed information to a reporting tool, or store it for later use.

以下是一些可以使用 Azure 串流分析的範例:Following are some examples where Azure Stream Analytics can be used:

  • 物聯網 (IoT) 感應器融合和即時分析裝置遙測Internet of Things(IoT) Sensor fusion and real-time analytics on device telemetry
  • Web 記錄/點選流分析Web logs/clickstream analytics
  • 車隊管理和自駕車的地理空間分析Geospatial analytics for fleet management and driverless vehicles
  • 高價值資產的遠端監視和預測性維護Remote monitoring and predictive maintenance of hi-value assets
  • 即時分析銷售點資料以控制庫存和偵測異常Real-time analytics on Point of Sale data for inventory control and anomaly detection

串流分析如何運作?How does Stream Analytics work?

要進行 Azure 串流分析必須先有串流資料的來源,這些資料可能是內嵌至 Azure 事件中樞、Azure IoT 中樞,或來自 Azure Blob 儲存體等資料存放區。Azure Stream Analytics starts with a source of streaming data that is ingested into Azure Event Hub, Azure IoT Hub or from a data store like Azure Blob Storage. 若要檢查資料流,您可以建立串流分析作業以指定會串流資料的輸入來源。To examine the streams, you create a Stream Analytics job that specifies the input source that streams data. 此作業也會指定轉換查詢,以定義如何尋找資料、模式或關聯性。The job also specifies a transformation query that defines how to look for data, patterns, or relationships. 轉換查詢會利用類似 SQL 的查詢語言,以便篩選、排序、彙總和聯結一段時間內的串流資料。The transformation query leverages a SQL-like query language that is used to filter, sort, aggregate, and join streaming data over a period of time. 在執行作業時,您可以調整事件排序選項,而在執行彙總作業時則可調整時間範圍的持續時間。When executing the job, you can adjust the event ordering options, and duration of time windows when performing aggregation operations.

在分析傳入的資料後,您可以指定所轉換資料的輸出,並可控制所要採取的動作以回應您所分析的資訊。After analyzing the incoming data, you specify an output for the transformed data, and you can control what to do in response to the information you've analyzed. 例如,您可以採取下列動作:For example, you can take actions like:

  • 將資料傳送至受監視的佇列,以觸發自訂工作流程下游。Send data to a monitored queue to trigger custom workflows downstream.
  • 將資料傳送至 Power BI 儀表板以獲得即時的視覺效果。Send data to Power BI dashboard for real-time visualization.
  • 將資料封存至其他 Azure 儲存體服務。Archive data to other Azure storage services.

下圖說明串流分析管線,您的串流分析作業可以使用所有輸入和輸出,也可以使用一組選定的輸入和輸出。The following image illustrates the Stream Analytics pipeline, Your Stream Analytics job can use all or a selected set of inputs and outputs. 下圖顯示如何將資料傳送至串流分析、加以分析並傳送,以進行儲存或呈現等其他動作:This image shows how data is sent to Stream Analytics, analyzed, and sent for other actions like storage, or presentation:

串流分析流程

主要功能和優點Key capabilities and benefits

Azure 串流分析的設計訴求是方便使用、具靈活性、可靠,以及適用於任何作業大小。Azure Stream Analytics is designed to be easy to use, flexible, reliable, and scalable to any job size. 其可跨多個資料中心以及主權雲端來使用。It is available across multiple datacenters as well as sovereign clouds. 下圖說明 Azure 串流分析的重要功能:Following image illustrates the key capabilities of Azure Stream Analytics:

串流分析的重要功能

輕鬆開始使用Ease of getting started

您可以輕鬆地開始使用 Azure 串流分析。Azure Stream Analytics is easy to get started. 只要按幾下就能連線到多個來源、接收並建立端對端管線。It only takes few clicks to connect to multiple sources, sinks and to create an end to end pipeline. 串流分析可以連線到 Azure 事件中樞Azure IoT 中樞來擷取串流資料。Stream Analytics can connect to Azure Event Hubs, Azure IoT Hub for streaming data ingestion. 它也可以連線到 Azure Blob 儲存體服務來擷取歷史資料。It can also connect to Azure Blob storage service to ingest historical data. 它可以結合事件中樞的資料與其他資料來源和處理引擎。It can combine data from event hubs with other data sources and processing engines. 作業輸入也可以包含靜態參考資料或變更緩慢的資料,而且您可以將串流資料加入到此參考資料,以執行查閱作業。Job input can also include reference data that is static or slow-changing data and you can join streaming data to this reference data to perform lookup operations.

串流分析可將作業輸出路由至許多儲存體系統,例如 Azure BlobAzure SQL DatabaseAzure Data Lake StoreAzure Cosmos DBStream Analytics can route job output to many storage systems such as Azure Blob, Azure SQL Database, Azure Data Lake Stores, or Azure Cosmos DB. 儲存後,您可以使用 Azure HDInsight 執行批次分析,也可以將輸出傳送至另一個服務 (例如事件中樞) 以供取用,或傳送至 Power BI 以使用 Power BI 串流 API 來即時呈現視覺效果。After storing, you can run batch analytics with Azure HDInsight or send the output to another service such as event hubs for consumption or to Power BI for real-time visualization by using Power Bi streaming API.

程式設計人員生產力Programmer Productivity

Azure 串流分析使用已擴增的簡單 SQL 架構查詢語言,搭配強大的時態性限制式即可分析移動中的資料。Azure Stream Analytics uses a simple SQL based query language that has been augmented with powerful temporal constraints to analyze data in motion. 若要定義作業轉換,您可以使用簡單的敘述性串流分析查詢語言,來撰寫使用簡單 SQL 建構的複雜時態性查詢和分析。To define job transformations, you use a simple, declarative Stream Analytics query language that lets you author complex temporal queries and analytics using simple SQL constructs. 串流分析查詢語言與 SQL 語言一致,只要熟悉 SQL 語言便可開始建立作業。Stream Analytics query language is consistent to the SQL language, familiarity with SQL language is sufficient to get started with creating jobs. 使用 Azure PowerShell、串流分析 Visual Studio 工具或 Azure Resource Manager 範本等開發人員工具,也可以建立作業。You can also create jobs by using developer tools like Azure PowerShell, Stream Analytics Visual Studio tools, or Azure Resource Manager templates. 使用開發人員工具可讓您離線開發轉換查詢,並使用 CI/CD 管線將作業提交至 Azure。Using developer tools allow you to develop transformation queries offline and use the CI/CD pipeline to submit jobs to Azure.

串流分析查詢語言提供了各式各樣的函式,以供分析和處理串流資料。The Stream Analytics query language offers a wide array of functions for analyzing and processing the streaming data. 此查詢語言支援簡單的資料操作、彙總函式到複雜的地理空間函式。This query language supports simple data manipulation, aggregation functions to complex geo-spatial functions. 您可以在入口網站中編輯查詢,然後使用從即時資料流所擷取出的範例資料來測試查詢。You can edit queries in the portal, and test them using sample data that is extracted from the live stream.

您可以透過定義和叫用其他函式來延伸查詢語言的功能。You can extend the capabilities of the query language by defining and invoking additional functions. 您可以在 Azure Machine Learning 服務定義函式呼叫以利用 Azure Machine Learning 解決方案,並整合 JavaScript 使用者定義的函式 (UDF) 或使用者定義的彙總以在串流分析查詢中執行複雜的計算。You can define function calls in the Azure Machine Learning service to take advantage of Azure Machine Learning solutions and integrate JavaScript user-defined functions (UDFs) or user-defined aggregates to perform complex calculations as part a Stream Analytics query.

完全受控Fully managed

Azure 串流分析是 Azure 上完全受控的無伺服器 (PaaS) 供應項目。Azure Stream Analytics is a fully managed serverless (PaaS) offering on Azure. 這表示您不必佈建任何硬體或管理叢集即可執行您的作業。Which means you don’t have to provision any hardware or manage clusters to run your jobs. Azure 串流分析會全權負責管理您的作業,它會負責在雲端設定複雜的計算叢集,並負責處理執行作業所需的效能微調。Azure Stream Analytics fully manages your job, by taking care of setting up complex compute clusters in the cloud and the performance tuning necessary to run the job. 與 Azure 事件中樞和 Azure IoT 中樞整合,可讓作業每秒擷取數百萬來自連線裝置、點擊資料流及記錄檔等項目的事件。Integration with Azure Event Hubs and Azure IoT Hub allows jobs to ingest millions of events per second coming from connected devices, clickstreams, and log files, to name a few. 透過事件中樞的分割功能,您可以將計算分割為邏輯步驟,而每個步驟都可以再進一步地分割來增加延展性。Using the partitioning feature of event hubs, you can partition computations into logical steps, each with the ability to be further partitioned to increase scalability.

低擁有權總成本Low Total Cost of Ownership

作為雲端服務,串流分析已進行成本最佳化。As a cloud service, Stream Analytics is optimized for cost. 不會有預付費用,您只需針對您耗用的串流單位和處理的資料量付費。There are no upfront costs involved, you only pay for the streaming units you consume, and the amount of data processed. 不必做出承諾或佈建叢集。There is no commitment or cluster provisioning required. 您可以根據您的商務需求相應增加或相應減少串流作業。You can scale the job up or down your steaming jobs based on your business needs.

可靠性Reliability

作為受控服務,串流分析針對事件處理提供 99.9% 可用性的保證、協助防止資料遺失,並提供商務持續性。如需詳細資料,請參閱串流分析 SLA 頁面。As a managed service, Stream Analytics guarantees event processing with a 99.9% availability, helps prevent data loss, and provides business continuity refer to the Stream Analytics SLA page for more details. 串流分析每秒可以處理數百萬個事件,並可低延遲地傳遞結果。Stream Analytics can process millions of events every second and it can deliver results with low latency. 串流分析可保證僅只一次的事件處理,以及至少一次的事件傳遞。Stream Analytics guarantees exactly once event processing and at least once delivery of events. 其具有可在事件傳遞失敗時進行復原的內建功能。It has built-in recovery capabilities in case the delivery of an event fails. 串流分析可在內部維護作業的狀態,您可以從作業的最後一次輸出時間開始作業,藉由永遠提供相同的結果來提供可重複的結果。Stream Analytics can internally maintain the state of your job, you can start a job from its last output time, provides repeatable results by providing same results all the time. 串流分析的這項功能可讓您及時回頭檢視,並在執行根本原因分析時調查計算方式。This feature of Stream Analytics enables you to go back in time and investigate computations when doing root-cause analysis.

效能Performance

為了獲得高效能,Azure 串流分析已經過最佳化,可以處理串流資料並執行記憶體內部計算。Azure Stream Analytics is optimized for high performance, it can process streaming data and perform in memory computations. 它可讓您相應增加或相應減少以處理即時和複雜的事件處理應用程式。It allows you to scale-up or scale-down to handle real-time and complex event processing applications. 串流分析藉由分割來支援效能。Stream Analytics supports performance by partitioning. 您可以在多個串流節點上平行處理和執行複雜的查詢。A complex query can be parallelized and executed on multiple streaming nodes.

後續步驟Next steps

您現在已大致了解 Azure 串流分析。You now have an overview of Azure Stream Analytics. 接下來,您可以深入了解並建立您的第一個串流分析作業:Next, you can dive deep and create your first Stream Analytics job: