分析 Bot 的遙測資料Analyze your bot's telemetry data

適用于: SDK v4APPLIES TO: SDK v4

分析 Bot 行為Analyzing Bot behavior

下列查詢集合可用來分析 Bot 行為。The following collection of queries can be used to analyze bot behavior. 您可以使用集合,在 Azure 監視器 Log Analytics 中撰寫自訂查詢,以及建立監視和 PowerBI 視覺效果儀表板。You can use the collection to author custom queries in Azure Monitor Log Analytics and to create monitoring and PowerBI visualization dashboards.

PrerequisitesPrerequisites

對下列概念有基本瞭解很有幫助:It is helpful to have a basic understanding of the following concepts:

提示

如果您使用 Power Virtual Agents編輯器等工具建立您的 bot,您會想要使用每個查詢的適應性對話方塊版本(如果有的話)。If your create your bot using tools such as Power Virtual Agents or Composer, you will want to use the Adaptive Dialog version of each query when available.

儀表板Dashboards

Azure 儀表板提供絕佳的方式,讓您檢視和共用查詢所產生的資訊。Azure Dashboards offer a great way to view and share the information generated from your queries. 您可以建置自訂儀表板,藉由將查詢與新增至儀表板的圖格產生關聯,協助監視 Bot 活動。You can build custom dashboards to help monitor your bots activity by associating your queries with the tiles that you add to your dashboard. 如需儀表板的詳細資訊,以及如何將查詢與其建立關聯,請參閱建立和共用 Log Analytics 資料的儀表板For more information on dashboards and how to associate your queries with them, see Create and share dashboards of Log Analytics data. 本文的其餘部分說明您可能發現適用於監視 Bot 行為的一些查詢範例。The remainder of this article shows examples of some of the queries that you may find useful in monitoring your bots behavior.

範例 Kusto 查詢Example Kusto queries

注意

建議您針對本文中的所有查詢,在不同的維度 (例如期間、通道和地區設定) 上進行轉移。It is recommended to pivot on different dimensions such as period, channel, and locale for all queries in this article.

各期間的使用者數目Number of users per period

這個範例會產生一個折線圖,其中顯示過去 14 天內每天與 Bot 通訊的相異使用者人數。This example results in a line chart that shows how many distinct users communicated with your bot per day for the past 14 days. 您可以將不同的值指派給 queryStartDatequeryEndDateinterval 變數,輕鬆地變更時間週期。The time period can be easily changed by assigning different values to the queryStartDate, queryEndDate and interval variables.

重要

只有當使用者是已經過驗證的使用者,而且結果也可能取決於通道功能時,您才會在此查詢中取得唯一使用者的正確計數。You will only get a correct count of unique users in this query if they are authenticated users, and the results may also depend on the channel capabilities.

// number of users per period
let queryStartDate = ago(14d);
let queryEndDate = now();
let groupByInterval = 1d;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| summarize uc=dcount(user_Id) by bin(timestamp, groupByInterval)
| render timechart

提示

Kusto summarize 運算子用於產生可彙總輸入資料表內容的資料表。The Kusto summarize operator is used to produce a table that aggregates the content of the input table.

Bin 函式是 Kusto 純量函式,其與 summarize operator 搭配使用時,會將查詢結果分組到指定的值。The Bin function is a Kusto scalar function that when used in conjunction with the summarize operator will group the query results into the specified value. 在上述範例中,這是依日期分組,Kusto 也會接受 h=小時、m=分鐘、s=秒、ms=毫秒、微秒=微秒。In the above example, this is grouped by day, Kusto also will accept h=hours, m=minutes, s=seconds, ms=milliseconds, microsecond=microseconds.

render 運算子 可讓您輕鬆地轉譯圖表,例如 timechart,這是一個折線圖,其中 X 軸是 datetime,而任何其他數值資料行都可以用於 Y 軸。The render operator enables you to easily render charts, such as the timechart, a line chart where the x-axis is a datetime and any other numeric column can be used for the y-axis. 即使您的資料並未每次指定,其也會讓 X 軸自動保持良好間距。It automatically keeps the x-axis spaced nicely even if your data doesn't have every time specified. 如果未使用 render 陳述式,則預設為 tableIf no render statement is used, it defaults to table.

每一期間的使用者數查詢結果範例Sample number-of-users-per-period query results

各期間的使用者數目

各期間的活動Activity per period

這個範例說明如何測量每個所需維度的活動量,而不論這是過去 14 天內每天的交談、對話或訊息數目的計數。This example illustrates how to measure the volume of activity per desired dimension, whether that be a count of the number of conversations, dialogs, or messages per day for the past 14 days. 您可以將不同的值指派給 querystartdatequeryEndDateinterval 變數,輕鬆地變更時間週期。The time period can be easily changed by assigning different values to the querystartdate, queryEndDate and interval variables. 所需的維度是由 extend 下列範例中的子句所定義, metric 可以設定為 InstanceIdDialogIdactivityIdThe desired dimension is defined by the extend clause in the following example, metric can be set to either InstanceId, DialogId or activityId.

計量 指派給您要顯示的維度:Assign metric to the dimension that you want to display:

  • InstanceId 可測量 交談的數目InstanceId measures the number of Conversations
  • DialogId 可測量 對話的數目DialogId measures the number of Dialogs
  • ActivityId 可測量 訊息的數目ActivityId measures the number of Messages
// Measures the number of activity's (conversations, dialogs, messages) per period.
let queryStartDate = ago(14d);
let queryEndDate = now();
let groupByInterval = 1d;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend InstanceId = tostring(customDimensions['InstanceId'])
| extend DialogId = tostring(customDimensions['DialogId'])
| extend ActivityId = tostring(customDimensions['activityId'])
| where DialogId != '' and  InstanceId != '' and user_Id != ''
| extend metric = InstanceId // DialogId or ActivityId
| summarize Count=dcount(metric) by  bin(timestamp, groupByInterval)
| order by Count desc nulls last
| render timechart

提示

Kusto extend 運算子用於建立計算結果欄並將其附加至結果集。The Kusto extend operator is used to create calculated columns and append them to the result set.

每一期間的範例活動查詢結果Sample activity-per-period query results

各期間的活動

各期間各使用者的活動Activity per user per period

這個範例示範如何計算每個期間每位使用者的活動數目。This example demonstrates how to count the number of activities per user per period. 其會示範如何在 各期間的活動 查詢中向下切入,以將焦點放在每個期間每位使用者的活動。This demonstrates drilling down in the activity per period query to focus on the activity per user per period. 這些活動包括對話、交談或訊息。The activities include dialogs, conversations or messages. 這有助於測量使用者與 Bot 的互動,並可協助判斷潛在問題,例如:This helps to measure user interaction with your bot and can help in determining potential problems, for example:

  • 由單一使用者進行許多活動的天數可能表示攻擊或測試Days with lots of activity by a single user may mean attack or test
  • 少量互動的天數可能表示服務健康情況問題Days with little interaction may indicate service health issues

提示

您可以依照 user_Id 移除,以取得可隨時間和對話、訊息或交談上進行轉移的一般 Bot 活動量。You can remove by user_Id to get the general bot activity volume which can be pivoted on time and dialogs, messages, or conversations.

// number of users per period per dialogs
let queryStartDate = ago(14d);
let queryEndDate = now();
let interval = 6h;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend InstanceId = tostring(customDimensions['InstanceId'])
| extend DialogId = tostring(customDimensions['DialogId'])
| extend ActivityId = tostring(customDimensions['activityId'])
| where DialogId != '' and InstanceId != '' and user_Id != ''
| extend metric = ActivityId // InstanceId // DialogId // or InstanceId for conversation count
| summarize Count=dcount(metric) by user_Id, bin(timestamp, groupByInterval)
| order by Count desc nulls last

每位使用者每個使用者的範例活動查詢結果Sample activity-per-user-per-period query results

user_Iduser_Id timestamptimestamp CountCount
User-8107ffd2User-8107ffd2 2019-09-03T00:00:00Z2019-09-03T00:00:00Z 1414
User-75f2cc8fUser-75f2cc8f 2019-08-30T00:00:00Z2019-08-30T00:00:00Z 1313
User-75f2cc8dUser-75f2cc8d 2019-09-03T00:00:00Z2019-09-03T00:00:00Z 1313
User-3060aadaUser-3060aada 2019-09-03T00:00:00Z2019-09-03T00:00:00Z 1010

對話完成Dialog completion

一旦設定對話的遙測用戶端,對話 (及其子系) 就會發出一些預設遙測資料,例如 [已開始] 和 [已完成] 。Once you set the telemetry client for a dialog, the dialog (and its children) will emit some default telemetry data, such as started and completed. 此範例可用來測量相對於 [已開始] 對話的 [已完成] 對話。This example can be used to measure the completed dialogs relative to started dialogs. 如果已開始對話數目大於已完成的數目,則表示您有某些使用者並未完成對話流程。If the number of dialogs started is greater than the number completed, some of your users are not completing the dialog flow. 這可用來作為識別和疑難排解任何潛在對話邏輯的起點。This can be used as a starting point in identifying and troubleshooting any potential dialog logic. 其也可用來識別較熱門和較不頻繁的對話。It can also be used to identify the more popular and less frequented dialogs.

提示

如果您使用 Power Virtual Agents編輯器等工具建立您的 bot,您會想要使用每個查詢的適應性對話方塊版本。If your create your bot using tools such as Power Virtual Agents or Composer, you will want to use the adaptive dialog version of each query.

瀑布式對話方塊完成Waterfall dialog completion

// % Completed Waterfall Dialog: shows completes relative to starts
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| where name=="WaterfallStart"
| extend DialogId = customDimensions['DialogId']
| extend InstanceId = tostring(customDimensions['InstanceId'])
| join kind=leftouter (
    customEvents
    | where name=="WaterfallComplete"
    | extend InstanceId = tostring(customDimensions['InstanceId'])
  ) on InstanceId
| summarize started=countif(name=='WaterfallStart'), completed=countif(name1=='WaterfallComplete') by tostring(DialogId)
| where started > 100  // filter for sample
// Show starts vs. completes
| project tostring(DialogId), started, completed
| order by started desc, completed asc  nulls last
| render barchart  with (kind=unstacked, xcolumn=DialogId, ycolumns=completed, started, ysplit=axes)

提示

Kusto join 運算子藉由比對每個資料表中所指定資料行的值,合併兩個資料表的資料列,以形成新的資料表。The Kusto join operator is used to merge the rows of two tables to form a new table by matching values of the specified column(s) from each table.

project 運算子用於選取您要在輸出中顯示的欄位。The project operator is used to select the fields that you want to show up in your output. 類似於可新增欄位的 extend operatorproject operator 可以從現有的欄位集中選擇,或新增欄位。Similar to the extend operator that adds a new field, the project operator can either choose from the existing set of fields or add a new field.

自動調整對話方塊已啟動和完成Adaptive dialogs started and completed

// % Completed adaptive dialog: shows completes relative to starts. This type is the default dialog type when using Power Virtual Agents or Composer. 
customEvents
| where name=="AdaptiveDialogStart" or name == "AdaptiveDialogComplete"
| extend DialogId = tostring(customDimensions['DialogId'])
| summarize started=countif(name=='AdaptiveDialogStart'), completed=countif(name=='AdaptiveDialogComplete') by DialogId
| project DialogId, started, completed
| order by started desc, completed asc nulls last
| render barchart with (kind=unstacked, xcolumn=DialogId, ycolumns=completed, started, ysplit=axes)

範例對話方塊-完成查詢結果Sample dialog-completion query results

對話完成

對話未完成Dialog incompletion

此範例可用來計算在指定的時間週期內,已經開始但因為取消或放棄而從未完成的對話流程數目。This example can be used to count the number of dialog flows that started but never completed due to cancellation or abandonment during the specified time period. 您可將其用來檢閱不完整的對話,並檢查對話是否因為使用者分心而遭主動取消,或只是因為使用者分心或失去興趣而被放棄。You can use it to review incomplete dialogs and examine whether they were actively cancelled due to user confusion or simply abandoned due to user distraction or loss of interest.

瀑布式對話未完成Waterfall dialogs not completed

// Show incomplete dialogs when using waterfall dialogs.
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents 
| where timestamp > queryStartDate 
| where timestamp < queryEndDate
| where name == "WaterfallStart" 
| extend DialogId = customDimensions['DialogId']
| extend instanceId = tostring(customDimensions['InstanceId'])
| join kind=leftanti (
  customEvents
  | where name == "WaterfallComplete" 
  | extend instanceId = tostring(customDimensions['InstanceId'])
  ) on instanceId
| summarize cnt=count() by  tostring(DialogId)
| order by cnt
| render barchart

未完成適應性對話方塊Adaptive dialogs not completed

// Show incomplete dialogs for adaptive dialogs; this type is the default dialog type when using Power Virtual Agents or Composer.
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where name == "AdaptiveDialogStart"
| extend DialogId = tostring(customDimensions['DialogId'])
| join kind=rightanti (
customEvents
| where name == "AdaptiveDialogComplete"
| extend DialogId = tostring(customDimensions['DialogId'])
) on name, DialogId
| summarize cnt=count() by DialogId
| order by cnt
| render barchart

提示

Kusto order 運算子 (與 sort operator 相同) 用於將輸入資料表的資料列分類成依一或多個資料行排序。The Kusto order operator (Same as the sort operator) is used to sort the rows of the input table into order by one or more columns. 注意:如果您想要從任何查詢的結果中排除 null 值,您可以在 where 陳述式中將其篩選出來,例如您可以新增 "and isnotnull(Timestamp)",或在開頭或結尾傳回 null 值,將 nulls firstnulls first 新增至 order 陳述式的結尾。Note: If you want to exclude null values from the results of any query, you can filter them out in your where statement, for example you could add "and isnotnull(Timestamp)", or to return null values at the beginning or end, add the nulls first or nulls first to the end of the order statement.

範例對話方塊-incompletion 查詢結果Sample dialog-incompletion query results

摘要作業的查詢結果範例

對話序列向下切入Dialog Sequence Drill Down

交談中對話的瀑布開始/步驟/完成Waterfall start/step/complete for dialog in conversation

這個範例顯示對話步驟的序列 (並依交談 (instanceId) 分組)。This example shows the sequence of dialog steps, grouped by conversation (instanceId). 這有助於判斷哪些步驟導致對話中斷。This can be useful in determining which steps lead to dialog interruption.

執行此查詢,輸入所需的值來 DialogId 取代 <SampleDialogId>The run this query, enter the value of the desired DialogId in place of <SampleDialogId>

// Drill down: Show waterfall start/step/complete for specific dialog
let queryStartDate = ago(14d);
let queryEndDate = now();
let DialogActivity=(dlgid:string) {
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend DialogId = customDimensions['DialogId']
| extend StepName = customDimensions['StepName']
| extend InstanceId = customDimensions['InstanceId']
| where DialogId == dlgid
| project timestamp, name, StepName, InstanceId
| order by tostring(InstanceId), timestamp asc
};
// For example see SampleDialogId behavior
DialogActivity("<SampleDialogId>")

提示

此查詢是使用查詢定義的函式所撰寫,這是在單一查詢範圍內定義和使用的使用者定義函式,而且是透過 let 陳述式定義的。This query was written using a query-defined function, which is a user-defined function that is defined and used within the scope of a single query, and is defined through a let statement. 撰寫此查詢時不需使用 query-defined functionThis query written without the use of the query-defined function:

let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend DialogId = customDimensions['DialogId']
| extend StepName = customDimensions['StepName']
| extend InstanceId = customDimensions['InstanceId']
| where DialogId == "<SampleDialogId>"
| project timestamp, name, StepName, InstanceId
| order by tostring(InstanceId), timestamp asc
查詢結果範例Sample query results
timestamptimestamp namename StepNameStepName InstanceIdInstanceId
2019-08-23T20:04...2019-08-23T20:04... WaterfallStartWaterfallStart nullnull ...79c0f03d8701...79c0f03d8701
2019-08-23T20:04...2019-08-23T20:04... WaterfallStepWaterfallStep GetPointOfInterestLocationsGetPointOfInterestLocations ...79c0f03d8701...79c0f03d8701
2019-08-23T20:04...2019-08-23T20:04... WaterfallStepWaterfallStep ProcessPointOfInterestSelectionProcessPointOfInterestSelection ...79c0f03d8701...79c0f03d8701
2019-08-23T20:04...2019-08-23T20:04... WaterfallStepWaterfallStep GetRoutesToDestinationGetRoutesToDestination ...79c0f03d8701...79c0f03d8701
2019-08-23T20:05...2019-08-23T20:05... WaterfallStepWaterfallStep ResponseToStartRoutePromptResponseToStartRoutePrompt ...79c0f03d8701...79c0f03d8701
2019-08-23T20:05...2019-08-23T20:05... WaterfallComplete 1WaterfallComplete 1 nullnull ...79c0f03d8701...79c0f03d8701
2019-08-28T23:35...2019-08-28T23:35... WaterfallStartWaterfallStart nullnull ...6ac8b3211b99...6ac8b3211b99
2019-08-28T23:35...2019-08-28T23:35... WaterfallStep 2WaterfallStep 2 GetPointOfInterestLocationsGetPointOfInterestLocations ...6ac8b3211b99...6ac8b3211b99
2019-08-28T19:41...2019-08-28T19:41... WaterfallStartWaterfallStart nullnull ...8137d76a5cbb...8137d76a5cbb
2019-08-28T19:41...2019-08-28T19:41... WaterfallStep 2WaterfallStep 2 GetPointOfInterestLocationsGetPointOfInterestLocations ...8137d76a5cbb...8137d76a5cbb
2019-08-28T19:41...2019-08-28T19:41... WaterfallStartWaterfallStart nullnull ...8137d76a5cbb...8137d76a5cbb

1 已完成1 Completed

放棄 22 Abandoned

解讀:使用者似乎放棄在 GetPointOfInterestLocations 步驟的交談。Interpretation: Users seem to abandon the conversation at the GetPointOfInterestLocations step.

注意

瀑布式對話會執行序列 (開始、多個步驟、完成)。Waterfall dialogs execute a sequence (start, multiple steps, complete). 如果序列顯示開始但未完成,表示對話因為使用者放棄或取消對話而中斷。If a sequence shows start with no complete, it means the dialog was interrupted either due to user abandoning or canceling the dialog. 在此詳細的分析中,可以查看此行為 (請參閱已完成與已放棄的步驟)。In this detailed analysis, one can see this behavior (see completed vs. abandoned steps).

瀑布式開始/步驟/完成/取消步驟彙總總計Waterfall start/step/complete/cancel steps aggregate totals

這個範例會顯示下列各項的彙總總計:對話序列開始的總次數、瀑布式步驟合計總數、成功完成的數目、已取消的數目,而 WaterfallStartWaterfallComplete 加上 WaterfallCancel 的合計總數之間的差異會提供放棄總數。This example shows the aggregate totals of the total number of times that a dialog sequence was started, the combined total number of waterfall steps, how many were successfully completed, how many were canceled and the difference between WaterfallStart and the combined total of WaterfallComplete plus WaterfallCancel will give you the total number abandoned.

// Drill down: Aggregate view of waterfall start/step/complete/cancel steps totals for specific dialog
let queryStartDate = ago(14d);
let queryEndDate = now();
let DialogSteps=(dlgid:string) {
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend DialogId = customDimensions['DialogId']
| where DialogId == dlgid
| project name
| summarize count() by name
};
// For example see SampleDialogId behavior
DialogSteps("<SampleDialogId>")
範例瀑布-匯總查詢結果Sample waterfall-aggregate query results
namename 計數count
WaterfallStartWaterfallStart 2121
WaterfallStepWaterfallStep 4747
WaterfallCompleteWaterfallComplete 1111
WaterfallCancelWaterfallCancel 11

轉譯:對話順序的21個調用,只有11個已完成,已放棄9個,且使用者取消了一個。Interpretation: Of 21 invocations of dialog sequence, only 11 has completed, 9 were abandoned, and one was cancelled by the user.

對話中的平均持續時間Average duration in dialog

這個範例會測量使用者在指定對話中所花費的平均時間量。This example measures the average amount of time users spend in a given dialog. 對話所用時間冗長可能暗示有簡化的機會。A long time spend in a dialog may suggest opportunities to simplify.

// Average dialog duration
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| where name=="WaterfallStart"
| extend DialogId = customDimensions['DialogId']
| extend instanceId = tostring(customDimensions['InstanceId'])
| join kind=leftouter (customEvents | where name=="WaterfallCancel" | extend instanceId = tostring(customDimensions['InstanceId'])) on instanceId
| join kind=leftouter (customEvents | where name=="WaterfallComplete" | extend instanceId = tostring(customDimensions['InstanceId'])) on instanceId
| extend duration = case(not(isnull(timestamp1)), timestamp1 - timestamp,
not(isnull(timestamp2)), timestamp2 - timestamp, 0s) // Abandoned are not counted. Alternate: now()-timestamp)
| extend seconds = round(duration / 1s)
| summarize AvgSeconds=avg(seconds) by tostring(DialogId)
| order by AvgSeconds desc nulls last
| render barchart with (title="Duration in Dialog")

範例平均-持續時間查詢結果Sample average-duration query results

對話方塊持續時間的查詢結果範例

對話中的平均步驟Average steps in dialog

這個範例會依平均值、最小值、最大值和標準差的計算,顯示每個執行的對話「長度」。This example shows each executed dialogs "length" as calculated by average, min, max and standard deviation. 這有助於分析對話品質。This can help analyze dialog quality. 例如:For example:

  • 應針對具有許多步驟的對話,評估其簡化機會Dialogs with a lot of steps should be evaluated for simplification opportunities
  • 最小值/最大值/平均值之間有很大落差的對話可能表示使用者在嘗試完成工作時停滯。Dialogs with a wide gap between min/max/average could mean that users get stalled trying to complete the tasks. 您可能需要評估有較短路徑來完成工作的可能性,或降低對話複雜度的方式。You may need to evaluate the possibility of there being shorter paths to complete the tasks, or ways to reduce the dialog complexity.
  • 標準差很大的對話暗示複雜路徑或中斷體驗 (放棄/取消)Dialogs with a large standard-deviation suggest complex paths or broken experience (abandon/cancel)
  • 具有非常少步驟的對話可能因為其從不完成以致如此。Dialogs with very few steps may be so because they were never completed. 分析完成/放棄率可能有助於進行這項判斷。Analyzing the completion/abandonment rates may help to make that determination.
// min/max/std/avg steps per dialog
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend DialogId = tostring(customDimensions['DialogId'])
| extend StepName = tostring(customDimensions['StepName'])
| extend InstanceId = tostring(customDimensions['InstanceId'])
| where name == "WaterfallStart" or  name == "WaterfallStep" or  name == "WaterfallComplete"
| order by InstanceId, timestamp asc
| project timestamp, DialogId, name, InstanceId, StepName
| summarize cnt=count() by InstanceId, DialogId
| summarize avg=avg(cnt), minsteps=min(cnt),maxsteps=max(cnt), std=stdev(cnt) by DialogId
| extend avgsteps = round(avg, 1)
| extend avgshortbysteps=maxsteps-avgsteps
| extend avgshortbypercent=round((1.0 - avgsteps/maxsteps)*100.0, 1)
| project DialogId, avgsteps, minsteps, maxsteps, std, avgshortbysteps, avgshortbypercent
| order by std desc nulls last

範例平均-步驟查詢結果Sample average-steps query results

對話識別碼Dialog Id 平均步驟avg steps 最少步驟min steps 最多步驟max steps stdstd avg short by stepsavg short by steps avg short by percentavg short by percent
FindArticlesDialogFindArticlesDialog 6.26.2 22 77 2.042.04 0.80.8 11.4%11.4%
CreateTicketCreateTicket 4.34.3 22 55 1.51.5 0.70.7 14%14%
CheckForCurrentLocationCheckForCurrentLocation 3.93.9 22 55 1.411.41 1.11.1 22%22%
BaseAuthBaseAuth 3.33.3 22 44 1.031.03 0.70.7 17.5%17.5%
上架onboarding 2.72.7 22 44 0.940.94 1.31.3 32.5%32.5%

__解讀:例如,FindArticlesDialog 的範圍介於最小值/最大值之間,而且應該進行調查,並可能會重新設計和最佳化。__Interpretation: For example, FindArticlesDialog has a wide spread between min/max and should be investigated and possibly redesigned & optimized.

依活動計量的通道活動Channel activity by activity metric

這個範例會測量 Bot 在指定期間內經由每個通道收到的活動量。This example measures the amount of activity your bot receives per channel in the given period. 其做法是計算下列任何一個計量:傳入訊息、使用者、交談或對話。It does this by counting any one of the following metrics: incoming messages, users, conversations or dialogs. 這適用於服務健康情況分析,或用來測量通道的熱門程度。This can be useful for service health analysis or to measure a channels popularity.

// number of metric: messages, users, conversations, dialogs by channel
let queryStartDate = ago(14d);
let queryEndDate = now();
let groupByInterval = 1d;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| extend InstanceId = tostring(customDimensions['InstanceId'])
| extend DialogId = tostring(customDimensions['DialogId'])
| extend ActivityId = tostring(customDimensions['activityId'])
| extend ChannelId = tostring(customDimensions['channelId'])
| where DialogId != '' and  InstanceId != '' and user_Id != ''
| extend metric = user_Id // InstanceId or ActivityId or user_Id
| summarize Count=count(metric) by  ChannelId, bin(timestamp, groupByInterval)
| order by Count desc nulls last
| render barchart with (title="Users", kind=stacked) // or Incoming Messages or Conversations or Users

提示

您可能想要考慮嘗試下列變化:You may want to consider trying these variations:

  • 在不含時間戳記值區的情況下執行查詢: bin(timestamp, groupByInterval)Run the query without the timestamp bucketing: bin(timestamp, groupByInterval).
  • 您也可以針對所有使用者事件活動,使用不同使用者的 dcountcountYou can also use dcount for distinct users vs count for all user event activities. 這也適用於重複使用者。This also works for repeat users.

範例頻道-活動-依活動查詢結果Sample channel-activity-by-activity query results

通道使用方式的範例查詢結果

解讀:模擬器測試曾是最熱門的,但在我們上線之後,DirectLineSpeech 成為最熱門的通道。Interpretation: Emulator testing used to be most popular but once we went live, DirectLineSpeech, is the most popular channel.

依熱門程度的意圖總計Total Intents by popularity

這個範例適用於已啟用 LUIS 的 Bot。This example applies to LUIS enabled bots. 其會依熱門程度顯示所有意圖的摘要,以及對應的意圖偵測確實性分數。It shows a summary of all intents by popularity, and corresponding intent detection certainty score.

  • 實際上,每個計量的檢視都應該都是分開的。In practice, the view should separated for each metric.
  • 熱門的意圖路徑應針對使用者體驗進行最佳化。Popular intent paths should be optimized for user experience.
  • 低平均分數表示辨識度不佳和可能遺漏實際的使用者意圖。Low average scores indicate poor recognition & possible missing actual user intent.
// show total intents
let queryStartDate = ago(14d);
let queryEndDate = now();
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| where name startswith "LuisResult"
| extend intentName = tostring(customDimensions['intent'])
| extend intentScore = todouble(customDimensions['intentScore'])
| summarize ic=count(), ac=avg(intentScore)*100 by intentName
| project intentName, ic, ac
| order by ic desc nulls last
| render barchart with (kind=unstacked, xcolumn=intentName, ycolumns=ic,ac, title="Intents Popularity")

範例意圖-熱門查詢結果Sample intents-by-popularity query results

意圖熱門程度的查詢結果範例

解讀:例如,最熱門的意圖,偵測到確認的平均信賴度只有 23%。Interpretation: For example the most popular intent, confirm is detected only with 23% confidence on average.

提示

Barchart 是 Kusto 查詢所提供的數十個選項之一。Barcharts are one of over a dozen options available with Kusto queries. 其他選項包括:anomalychart、areachart、columnchart、linechart、scatterchart。Some other options include: anomalychart, areachart, columnchart, linechart, scatterchart. 如需詳細資訊,請參閱 render 運算子主題。for more details see the render operator topic.

Bot 分析檢測的結構描述Schema of Bot Analytics Instrumentation

下表顯示 Bot 將用來記錄遙測資料的最常見欄位。The following tables show the most common fields that your bot will log telemetry data into.

一般信封General Envelope

Application Insights 檢測中的常見記錄分析欄位。Common log analytics fields in Application Insights instrumentation.

欄位Field 說明Description 範例值Sample Values
NAMEname 訊息類型Message type BotMessageSend、BotMessageReceived、LuisResult、WaterfallStep、WaterfallStart、SkillWebSocketProcessRequestLatency、SkillWebSocketOpenCloseLatency、WaterfallComplete、QnaMessage、WaterfallCancel、SkillWebSocketTurnLatency、AuthPromptValidatorAsyncFailureBotMessageSend, BotMessageReceived, LuisResult, WaterfallStep, WaterfallStart, SkillWebSocketProcessRequestLatency, SkillWebSocketOpenCloseLatency, WaterfallComplete, QnaMessage, WaterfallCancel, SkillWebSocketTurnLatency, AuthPromptValidatorAsyncFailure
customDimensionscustomDimensions SDK Bot 分析SDK Bot Analytics activityId = <id> ,activityType = message,channelId = 模擬器,fromId = <id> ,FromName = User,locale = En-us,recipientId = <id> ,recipientName = Bot,text = 尋找咖啡廳activityId=<id>, activityType=message, channelId=emulator, fromId=<id>, fromName=User, locale=en-us, recipientId=<id>, recipientName=Bot, text=find a coffee shop
timestamptimestamp 事件時間Time of event 2019-09-05T18:32:45.287082Z2019-09-05T18:32:45.287082Z
instance_Idinstance_Id 交談識別碼Conversation Id f7b2c416-a680-4b2c-b4cc-79c0f03d8711f7b2c416-a680-4b2c-b4cc-79c0f03d8711
operation_Idoperation_Id 回合識別碼Turn Id 084b2856947e3844a5a18a8476d99aaa084b2856947e3844a5a18a8476d99aaa
user_Iduser_Id 唯一通道使用者識別碼Unique channel user id emulator7c259c8e-2f47...emulator7c259c8e-2f47...
client_IPclient_IP 用戶端 IP 位址Client ip address 127.0.0.1 (可能因為隱私權區塊而不存在)127.0.0.1 (may be absent due to privacy block)
client_Cityclient_City 用戶端城市Client city 雷德蒙德 (如果偵測到,可能不存在)Redmond (if detected, may be absent)

自訂維度Custom Dimensions

大部分的 Bot 特定活動資料都會儲存在 customDimensions 欄位中。Most of bot specific activity data is stored in customDimensions field.

欄位Field 說明Description 範例值Sample Values
activityIdactivityId 訊息識別碼Message id <id>: 8da6d750-d00b-11e9-80e0-c14234b3bc2a<id>: 8da6d750-d00b-11e9-80e0-c14234b3bc2a
activityTypeactivityType 訊息類型Type of message message、conversationUpdate、event、invokemessage, conversationUpdate, event, invoke
channelIdchannelId 通道識別碼Channel identifier emulator、directline、msteams、webchatemulator, directline, msteams, webchat
fromIdfromId 來源識別碼From Identifier <id>
fromNamefromName 用戶端的使用者名稱Username from client John Bonham、Keith Moon、Steve Smith、Steve GaddJohn Bonham, Keith Moon, Steve Smith, Steve Gadd
地區設定locale 用戶端原點地區設定Client origin locale en-us、zh-cn、en-GB、de-de、zh-CNen-us, zh-cn, en-GB, de-de, zh-CN
recipientIdrecipientId 收件者識別碼Recipient identifier <id>
recipientNamerecipientName 收件者名稱Recipient name John Bonham、Keith Moon、Steve Smith、Steve GaddJohn Bonham, Keith Moon, Steve Smith, Steve Gadd
texttext 訊息中的文字Text in message 尋找咖啡廳find a coffee shop

自訂維度:LUISCustom Dimensions: LUIS

LUIS 檢測會將其資料儲存在下列 [自訂維度] 欄位中。LUIS instrumentation stores its data in the following Custom Dimensions fields.

欄位Field 說明Description 範例值Sample Values
意圖intent LUIS 偵測到的意圖LUIS detected intent pointOfInterestSkillpointOfInterestSkill
intentScoreintentScore LUIS 辨識分數LUIS recognition score 0.980.98
實體Entities LUIS 偵測到的實體LUIS detected entities FoodOfGrocery = [["coffee"]], KEYWORD= ["coffee shop"]FoodOfGrocery = [["coffee"]], KEYWORD= ["coffee shop"]
問題Question LUIS 偵測到的問題LUIS detected question 尋找咖啡廳find a coffee shop
sentimentLabelsentimentLabel LUIS 偵測到的情緒LUIS detected sentiment 正面positive

自訂維度:QnAMakerCustom Dimensions: QnAMaker

QnAMaker 檢測會將其資料儲存在下列 [自訂維度] 欄位中。QnAMaker instrumentation stores its data in the following Custom Dimensions fields.

提示

若要啟用個人資訊記錄(例如問題和解答),請在 QnA Maker 類別的函式中,將 [記錄個人資訊] 參數設定為 [true]。To enable logging of personal information like questions and answers, the log personal information parameter should be set to true in the constructor of the QnA Maker class.

欄位Field 說明Description 範例值Sample Values
問題question QnA 偵測到的問題QnA detected question 您可以做什麼?what can you do?
接聽answer QnA 解答QnA answer 您有問題,我可能有解答。You have questions, I may have answers.
articleFoundarticleFound QnAQnA truetrue
questionIdquestionId QnA 問題識別碼QnA question Id 488488
knowledgeBaseIdknowledgeBaseId QnA KB IdQnA KB Id 2a4936f3-b2c8-44ff-b21f-67bc413b97272a4936f3-b2c8-44ff-b21f-67bc413b9727
matchedQuestionmatchedQuestion 相符問題的陣列Array of matched questions ["Can you explain to me what your role is?","Can you tell me a bit about yourself?","Can you tell me about you?","could you help me","hmmm so what can you do?","how can you help me","How can you help me?","How can you help?","so how can i use you in my projects?","Talk to me about your capability","What are you capable of?",…["Can you explain to me what your role is?","Can you tell me a bit about yourself?","Can you tell me about you?","could you help me","hmmm so what can you do?","how can you help me","How can you help me?","How can you help?","so how can i use you in my projects?","Talk to me about your capability","What are you capable of?",…

另請參閱See Also