將 Azure 診斷記錄串流至事件中樞Stream Azure Diagnostic Logs to an event hub

您可以使用入口網站中內建的「匯出至事件中樞」選項,或透過 Azure PowerShell Cmdlet 或 Azure CLI 來啟用診斷設定中事件中樞授權規則識別碼的方式,以近乎即時的速度將 Azure 診斷記錄 串流至任何應用程式。Azure diagnostic logs can be streamed in near real time to any application using the built-in “Export to Event Hubs” option in the Portal, or by enabling the Event Hub Authorization Rule ID in a diagnostic setting via the Azure PowerShell Cmdlets or Azure CLI.

您可以使用診斷記錄和事件中樞執行的項目What you can do with diagnostics logs and Event Hubs

這裡有一些您可以使用診斷記錄串流功能的方法:Here are just a few ways you might use the streaming capability for Diagnostic Logs:

  • 將記錄串流至第三方的記錄與遙測系統 – 您可以將所有診斷記錄串流至單一事件中樞,以將記錄資料透過管道傳送給第三方的 SIEM 或記錄分析工具。Stream logs to 3rd party logging and telemetry systems – You can stream all of your diagnostic logs to a single event hub to pipe log data to a third-party SIEM or log analytics tool.

  • 將「最忙碌路徑」資料串流至 Power BI 以檢視服務健全狀況:您可以使用事件中樞、串流分析和 PowerBI,輕鬆地將診斷資料轉換為 Azure 服務之近乎即時的見解。View service health by streaming “hot path” data to Power BI – Using Event Hubs, Stream Analytics, and Power BI, you can easily transform your diagnostics data in to near real-time insights on your Azure services. 此文件文章提供絕佳概觀,說明如何設定事件中樞、使用串流分析處理資料,以及使用 Power BI 作為輸出This documentation article gives a great overview of how to set up Event Hubs, process data with Stream Analytics, and use Power BI as an output. 以下是設定診斷記錄的一些祕訣︰Here are a few tips for getting set up with diagnostic logs:

    • 當您勾選入口網站中的選項,或透過 PowerShell 進行啟用時,就會自動建立診斷記錄類別的事件中樞,因此您需要選取命名空間中名稱開頭為 insights- 的事件中樞。An event hub for a category of diagnostic logs is created automatically when you check the option in the portal or enable it through PowerShell, so you want to select the event hub in the namespace with the name that starts with insights-.

    • 下列 SQL 程式碼是您可以使用的範例串流分析查詢,能將所有記錄資料剖析至 Power BI 表格:The following SQL code is a sample Stream Analytics query that you can use to parse all the log data in to a Power BI table:

      SELECT
      records.ArrayValue.[Properties you want to track]
      INTO
      [OutputSourceName – the Power BI source]
      FROM
      [InputSourceName] AS e
      CROSS APPLY GetArrayElements(e.records) AS records
      
  • 建置自訂遙測及記錄平台 – 如果您已有自建遙測平台或正好在考慮建置一個,事件中樞所具備的高度可調整的發佈訂閱特質可讓您靈活擷取診斷記錄。Build a custom telemetry and logging platform – If you already have a custom-built telemetry platform or are just thinking about building one, the highly scalable publish-subscribe nature of Event Hubs allows you to flexibly ingest diagnostic logs. 請參閱此處的 Dan Rosanova 指南,以在全球級別的遙測平台中使用事件中樞See Dan Rosanova’s guide to using Event Hubs in a global scale telemetry platform here.

啟用診斷記錄的串流Enable streaming of diagnostic logs

您可以透過入口網站或使用 Azure 監視器 REST API,啟用以程式控制方式對診斷記錄進行串流的功能。You can enable streaming of diagnostic logs programmatically, via the portal, or using the Azure Monitor REST APIs. 無論如何,您所建立的診斷設定可讓您指定事件中樞命名空間,以及記錄類別和您需要傳送至命名空間的計量。Either way, you create a diagnostic setting in which you specify an Event Hubs namespace and the log categories and metrics you want to send in to the namespace. 針對您所啟用的每個記錄類別,會在命名空間中建立事件中樞。An event hub is created in the namespace for each log category you enable. 診斷記錄類別是一種資源可以收集的記錄類型。A diagnostic log category is a type of log that a resource may collect.

警告

啟用和串流來自計算資源 (例如,VM 或 Service Fabric) 的診斷記錄 需要一組不同的步驟Enabling and streaming diagnostic logs from Compute resources (for example, VMs or Service Fabric) requires a different set of steps.

事件中樞命名空間不一定要和資源發出記錄屬於相同的訂用帳戶,只要進行設定的使用者有這兩個訂用帳戶的適當 RBAC 存取權,而且這兩個訂用帳戶都屬於同一個 ADD 租用戶。The Event Hubs namespace does not have to be in the same subscription as the resource emitting logs as long as the user who configures the setting has appropriate RBAC access to both subscriptions and both subscriptions are part of the same AAD tenant.

注意

目前不支援透過診斷設定傳送多維度計量。Sending multi-dimensional metrics via diagnostic settings is not currently supported. 跨維度值所彙總的維度計量會匯出為扁平化單一維度計量。Metrics with dimensions are exported as flattened single dimensional metrics, aggregated across dimension values.

例如:可以依據每個佇列層級來瀏覽及繪製「事件中樞」上的「傳入訊息」計量。For example: The 'Incoming Messages' metric on an Event Hub can be explored and charted on a per queue level. 不過,當您透過診斷設定匯出時,計量將會呈現為事件中樞內所有佇列的所有內送郵件。However, when exported via diagnostic settings the metric will be represented as all incoming messages across all queues in the Event Hub.

使用入口網站串流診斷記錄Stream diagnostic logs using the portal

  1. 在入口網站中,瀏覽至 Azure 監視器,然後按一下 [診斷設定] In the portal, navigate to Azure Monitor and click on Diagnostic Settings

    Azure 監視器的監視區段

  2. 選擇性地依資源群組或資源類型篩選清單,然後按一下您要設定診斷設定的資源。Optionally filter the list by resource group or resource type, then click on the resource for which you would like to set a diagnostic setting.

  3. 如果您選取的資源上沒有任何設定,系統會提示您建立設定。If no settings exist on the resource you have selected, you are prompted to create a setting. 按一下「開啟診斷」。Click "Turn on diagnostics."

    新增診斷設定 - 無現有的設定

    如果資源上已有設定,您將會看此資源上已設定的設定清單。If there are existing settings on the resource, you will see a list of settings already configured on this resource. 按一下「新增診斷設定」。Click "Add diagnostic setting."

    新增診斷設定 - 現有的設定

  4. 為設定提供名稱並核取 [串流至事件中樞] 方塊,然後選取 [事件中樞命名空間]。Give your setting a name and check the box for Stream to an event hub, then select an Event Hubs namespace.

    新增診斷設定 - 現有的設定

    選取的命名空間將會是事件中樞的建立所在 (如果這是您第一次的串流診斷記錄) 或串流處理的目的地 (如果已存在將該記錄分類串流至此命名空間的資源),而原則會定義串流機制擁有的權限。The namespace selected will be where the event hub is created (if this is your first time streaming diagnostic logs) or streamed to (if there are already resources that are streaming that log category to this namespace), and the policy defines the permissions that the streaming mechanism has. 目前,將事件串流到中樞需要管理、傳送和接聽的權限。Today, streaming to an event hub requires Manage, Send, and Listen permissions. 您可以在入口網站的 [設定] 索引標籤下,為您的命名空間建立或修改事件中樞命名空間共用存取原則。You can create or modify Event Hubs namespace shared access policies in the portal under the Configure tab for your namespace. 若要更新其中一個診斷設定,用戶端必須擁有事件中樞授權規則的 ListKey 權限。To update one of these diagnostic settings, the client must have the ListKey permission on the Event Hubs authorization rule. 您也可以選擇性地指定事件中樞名稱。You can also optionally specify an event hub name. 如果您指定事件中樞名稱,記錄便會路由至該事件中樞,而非路由至每個記錄類別所新建的事件中樞。If you specify an event hub name, logs are routed to that event hub rather than to a newly created event hub per log category.

  5. 按一下 [儲存] 。Click Save.

過了幾分鐘之後,新的設定就會出現在此資源的設定清單中,而且只要一產生新的事件資料,就會立即將診斷記錄串流至該事件中樞。After a few moments, the new setting appears in your list of settings for this resource, and diagnostic logs are streamed to that event hub as soon as new event data is generated.

透過 PowerShell CmdletVia PowerShell Cmdlets

注意

本文已更新為使用新的 Azure PowerShell Az 模組。This article has been updated to use the new Azure PowerShell Az module. AzureRM 模組在至少 2020 年 12 月之前都還會持續收到錯誤 (Bug) 修正,因此您仍然可以持續使用。You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. 若要深入了解新的 Az 模組和 AzureRM 的相容性,請參閱新的 Azure PowerShell Az 模組簡介To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. 如需 Az 模組安裝指示,請參閱安裝 Azure PowerShellFor Az module installation instructions, see Install Azure PowerShell.

若要透過 Azure PowerShell Cmdlet 啟用串流功能,您可以使用 Set-AzDiagnosticSetting Cmdlet 搭配下列參數︰To enable streaming via the Azure PowerShell Cmdlets, you can use the Set-AzDiagnosticSetting cmdlet with these parameters:

Set-AzDiagnosticSetting -ResourceId [your resource ID] -EventHubAuthorizationRuleId [your Event Hub namespace auth rule ID] -Enabled $true

「事件中樞授權規則識別碼」是一個採用下列格式的字串︰{Event Hub namespace resource ID}/authorizationrules/{key name},例如 /subscriptions/{subscription ID}/resourceGroups/{resource group}/providers/Microsoft.EventHub/namespaces/{Event Hub namespace}/authorizationrules/RootManageSharedAccessKeyThe Event Hub Authorization Rule ID is a string with this format: {Event Hub namespace resource ID}/authorizationrules/{key name}, for example, /subscriptions/{subscription ID}/resourceGroups/{resource group}/providers/Microsoft.EventHub/namespaces/{Event Hub namespace}/authorizationrules/RootManageSharedAccessKey. 您目前無法在 PowerShell 中選取特定的事件中樞名稱。You cannot currently select a particular event hub name with PowerShell.

透過 Azure CLIVia Azure CLI

若要透過 Azure CLI 啟用串流,您可以使用 az monitor diagnostic-settings create 命令。To enable streaming via the Azure CLI, you can use the az monitor diagnostic-settings create command.

az monitor diagnostic-settings create --name <diagnostic name> \
    --event-hub <event hub name> \
    --event-hub-rule <event hub rule ID> \
    --resource <target resource object ID> \
    --logs '[
    {
        "category": <category name>,
        "enabled": true
    }
    ]'

您可以將字典新增至傳遞作為 --logs 參數的 JSON 陣列,從而將其他類別新增至診斷記錄。You can add additional categories to the diagnostic log by adding dictionaries to the JSON array passed as the --logs parameter.

--event-hub-rule 引數會使用與「事件中樞授權規則識別碼」相同的格式,如 PowerShell Cmdlet 的說明。The --event-hub-rule argument uses the same format as the Event Hub Authorization Rule ID as explained for the PowerShell Cmdlet.

我要如何透過事件中樞取用記錄檔資料?How do I consume the log data from Event Hubs?

此處是來自事件中樞的範例輸出資料:Here is sample output data from Event Hubs:

{
    "records": [
        {
            "time": "2016-07-15T18:00:22.6235064Z",
            "workflowId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA",
            "resourceId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA/RUNS/08587330013509921957/ACTIONS/SEND_EMAIL",
            "category": "WorkflowRuntime",
            "level": "Error",
            "operationName": "Microsoft.Logic/workflows/workflowActionCompleted",
            "properties": {
                "$schema": "2016-04-01-preview",
                "startTime": "2016-07-15T17:58:55.048482Z",
                "endTime": "2016-07-15T18:00:22.4109204Z",
                "status": "Failed",
                "code": "BadGateway",
                "resource": {
                    "subscriptionId": "df602c9c-7aa0-407d-a6fb-eb20c8bd1192",
                    "resourceGroupName": "JohnKemTest",
                    "workflowId": "243aac67fe904cf195d4a28297803785",
                    "workflowName": "JohnKemTestLA",
                    "runId": "08587330013509921957",
                    "location": "westus",
                    "actionName": "Send_email"
                },
                "correlation": {
                    "actionTrackingId": "29a9862f-969b-4c70-90c4-dfbdc814e413",
                    "clientTrackingId": "08587330013509921958"
                }
            }
        },
        {
            "time": "2016-07-15T18:01:15.7532989Z",
            "workflowId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA",
            "resourceId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA/RUNS/08587330012106702630/ACTIONS/SEND_EMAIL",
            "category": "WorkflowRuntime",
            "level": "Information",
            "operationName": "Microsoft.Logic/workflows/workflowActionStarted",
            "properties": {
                "$schema": "2016-04-01-preview",
                "startTime": "2016-07-15T18:01:15.5828115Z",
                "status": "Running",
                "resource": {
                    "subscriptionId": "df602c9c-7aa0-407d-a6fb-eb20c8bd1192",
                    "resourceGroupName": "JohnKemTest",
                    "workflowId": "243aac67fe904cf195d4a28297803785",
                    "workflowName": "JohnKemTestLA",
                    "runId": "08587330012106702630",
                    "location": "westus",
                    "actionName": "Send_email"
                },
                "correlation": {
                    "actionTrackingId": "042fb72c-7bd4-439e-89eb-3cf4409d429e",
                    "clientTrackingId": "08587330012106702632"
                }
            }
        }
    ]
}
元素名稱Element Name 描述Description
記錄數records 此承載中的所有記錄事件的陣列。An array of all log events in this payload.
timetime 事件發生的時間。Time at which the event occurred.
categorycategory 此事件的記錄檔分類。Log category for this event.
resourceIdresourceId 產生此事件之資源的資源識別碼。Resource ID of the resource that generated this event.
operationNameoperationName 作業名稱。Name of the operation.
levellevel 選擇性。Optional. 表示記錄事件層級。Indicates the log event level.
propertiesproperties 事件的屬性。Properties of the event.

您可以在這裡檢視支援串流至事件中樞的所有資源提供者清單。You can view a list of all resource providers that support streaming to Event Hubs here.

從計算資源中串流資料Stream data from Compute resources

您也可以使用 Windows Azure 診斷代理程式,從計算資源中串流診斷記錄。You can also stream diagnostic logs from Compute resources using the Windows Azure Diagnostics agent. 請參閱本文章了解如何設定。See this article for how to set that up.

後續步驟Next steps