Measuring chatbot outcomes

Important

Power Virtual Agents capabilities and features are now part of Microsoft Copilot Studio following significant investments in generative AI and enhanced integrations across Microsoft Copilot.

Some articles and screenshots may refer to Power Virtual Agents while we update documentation and training content.

Tracking conversation engagement and outcomes is crucial to measure the chatbot performance metrics and spot areas for improvements in the analytics dashboard.

Important

A Microsoft Copilot Studio conversation between an end-user and a chatbot generates one or multiple analytics sessions. All analytics sessions are either Engaged or Unengaged.

Understanding analytics sessions outcomes

Analytics sessions each have an outcome:

• Unengaged sessions always have an outcome of None.
• Engaged sessions have one of the following outcomes:

• Resolved,
• Escalated,
• Abandoned.

Engagement and outcomes

Conversation analytics sessions, their engagement status, and their outcomes are available as part of the analytics dashboard or in the ConversationTranscript table in Dataverse.

Engagement	Outcome	Definition
Unengaged	None (None)	All unengaged sessions have an outcome of None.
Engaged	Escalated (HandOff)	A session is considered escalated when the Escalate topic is triggered, or the Transfer Conversation node is reached.
Engaged	Resolved (Resolved)	A session is considered resolved when the End of Conversation topic is triggered and the user confirms that the interaction was a success (impliedSuccess = FALSE) or doesn't answer and lets the session time out (impliedSuccess = TRUE).
Engaged	Abandoned (Abandoned)	A session is considered abandoned when an engaged session ends and didn't reach a resolved or escalated state.

Properly ending conversations to measure outcomes

It’s important to end conversation with the End of Conversation topic, so that the end-user can confirm their issue was resolved or not (and potentially escalate).

When the bot confirms success, a CSAT survey is displayed to the end-user to capture a 0-5 score.

If you’re certain of the outcome of a conversation based on the topic logic, you can directly flag it as a confirmed success or redirect to the Escalate topic.

Tip

In the unified authoring canvas, in the code editor, you can set the conversationOutcome within an action. For example conversationOutcome: ResolvedConfirmed for confirmed success or conversationOutcome: ResolvedImplied for implied success.