Analyze copilot performance

  • 5 minutes

After a copilot is deployed and customers are interacting with it, statistics that are related to the copilot will become available. You can access this information through the Analytics tab in the side navigation pane. On this tab, you can find key performance indicators (KPIs) that show the volume of sessions that your copilot has handled, how effectively your copilot was able to engage users and resolve issues, escalation rates to human agents, and abandonment rates during conversations. You'll find customer satisfaction information at the KPI level and on the Customer Satisfaction tab.

You can also view detailed session history and transcripts by selecting Sessions from the Analytics tab. On the Sessions page, you can download a file with the full session transcript, which can be a helpful way for you to adjust the performance of your copilot and change the content in your topics to improve your copilot's efficiency.

Analyze copilot performance and usage

The Summary page gives you a broad overview of your copilot's performance. It uses AI technology to show you which topics have the greatest impact on escalation rate, abandon rate, and resolution rate. For more information, see the table under Summary charts.

Screenshot of analytics.

The Summary page includes various charts with graphical views of your copilot's KPIs:

For more information, see Analyze copilot performance and usage.

The Engagement rate drivers, Abandon rate drivers, and Resolution rate drivers charts use natural language understanding to group issues as topics. These charts show you the topics that have the most impact on the performance of your copilot.

Analyze customer satisfaction for Microsoft Copilot Studio copilots

The Customer Satisfaction page provides a detailed view of customer satisfaction (CSAT) survey data, including the average CSAT score over time and the topics that have the most impact on the CSAT score. The Customer Satisfaction page includes various charts with graphical views of your copilot's customer satisfaction indicators:

  • Customer satisfaction drivers chart - Uses AI to group related support cases as topics and then displays topics in order of their impact on customer satisfaction over the specified time period.

  • Scores over time chart - Provides a graphical view of the average CSAT score over the specified time period.

  • Average CSAT score - Provides a graphical view of the average of CSAT scores for sessions in which customers respond to an end-of-session request to take the survey.

  • CSAT survey response rate - Shows the number of CSAT surveys that were presented and the percentage of surveys that were completed.

For more information, see Analyze customer satisfaction.

Analyze topic usage in Microsoft Copilot Studio

The topic details page provides a view into the performance of individual topics and how they can be improved. You can display the topic details page by selecting the Detail link in one of the following charts on the Summary and Customer Satisfaction pages:

Screenshot of how escalation rate drivers are viewed.

The topic details page can also be displayed by opening an individual topic from the Topics page and selecting Analytics at the top of the page.

Screenshot of the analytics button.

The topic details page includes various charts with graphical views of a topic's KPIs:

  • Topic Summary charts - Summarize topic performance indicators for the specified time period and the percent change over the period.

  • Impact Summary charts - Summarize the impact of the topic on KPIs for the specified time period.

  • Topic Volume by Day chart - Provides a graphical view of the number of sessions for the topic over the specified time period.

For more information, see Analyze topics usage.

Analyze session information in Microsoft Copilot Studio

Having access to session information that is related to your copilot can help you identify potential changes that need to be made. By default, you can download up to seven days of copilot conversation transcript sessions from the past 30 days directly from the Microsoft Copilot Studio portal.

Session transcripts are available by going to Analytics > Sessions tab. On the Sessions tab, you'll be able to see all the different sessions that have been run for the copilot. A session represents a conversation that someone had with the copilot. If your copilot had a high number of sessions, they'll be broken down into multiple rows to help make managing the sessions easier. Each row will contain 2,500 sessions. You can select each row to download the session transcripts for the specified time frame.

The downloaded file contains the following information:

  • SessionID - A unique identifier for each session.

  • StartDateTime - Time at which the session started. Entries are sorted by this column in descending order.

  • InitialUserMessage - First message that is entered by the user.

  • TopicName - Name of the last authored topic that was triggered in this session.

  • ChatTranscript - Transcript of the session in the following format:

    • User says:" "; Copilot says:" "; structure

    • Conversation turns are separated by semicolons

    • Copilot says doesn't include the options that are presented to the user

  • SessionOutcome - Outcome of the session (Resolved, Escalated, Abandoned, Unengaged).

  • TopicId - A unique identifier of the last authored topic that was triggered in this session.

For more information, see Analyze session information.