Typical Root-cause analysis (RCA) on my content

This document will guide you through a series of investigations to determine the root cause of low CSAT ratings for your product or service. You will be provided with step-by-step instructions to narrow down a specific topic or set of topics that lower overall CSAT. The instructions will take you through the steps to drill down in each area (customers feedback, ratings and localization quality) to identify the issue's root cause using existing PBI and SkyEye reports.

Apply root cause analysis techniques to diagnose the sources of customer pain points

How do I find the articles that have most impact on CSAT of the service?

Individual pages with a high volume of not helpful votes negatively impacts your service CSAT and can be a root cause of low CSAT.

In this section, you will learn a technique for identifying a set of articles with a high number of PVs and helpful votes below 50%.

Step 1. Create a data set for investigation.

  • Navigate to the SkyEye Analytics Hub http://skyeyereports.cloudapp.net/ , open the Topics (Services) report, and display the “Services: Topic Drill Down” tab.
  • Select a service of your choice from the Service filter list.

NOTE: we will use “Active Directory” as an example

  • Click in the Summaries per topic section (to ensure this section got a focus) and expend filters on the right.
  • Set “Helpful%” is less than 50% and click “Apply filter” link.
  • Click “Summaries per topic” table and sort results on the “PageView” column descending.
  • At the top of the result set you will see articles with high PV count and helpful ratings below 50%
  • Click on “Detail” column to further investigate on details (feedback, CSAT, etc.) for any article.

Advanced option

You can export the results data set from the previous step into excel file to further identify how much overall CSAT is impacted by each article.

Step 1. Export data into excel file

  • Click the three dots icon in the upper-right corner of the “Summaries per topic” table, and select Export data.
  • Choose Export “Summarized data” option.

Save your data file as “Summary per topic”.xlsx

Step 2. Investigate the newly created data set (Summary per topic.xlsx)

In this step, you will learn a technique to identify a list of articles that are responsible for 50% of the “not helpful” ratings which lowers CSAT for your service.

  • In the “Summary per topic”.xlsx, Insert a new column to the left of “NotHelpful” column and name new column “NH%ToAllPortfolio”
  • Click on the first cell in the new column (C2) and type the formula =[NotHelpful]/SUM([NotHelpful])
  • Select “NH%ToAllPortfolio” and click “%” to format the column values
  • Filter on “NH%ToAllPortfolio” in descending order
  • Explore the top 50%

In our example with “Active Directory” Service there are eight articles that are contributing to 50% of the negative ratings. To investigate further the root cause of a low ratings for the list of articles you identified it’s possible to check the customer’s verbatim via Verbatims report

How do I find articles with the opportunity to improve the CSAT quickly?

Through the SkyEye User Comments search tool you can quickly identify articles with a high number of unique visitors and low CSAT ratings. In addition, you can find a low CSAT root cause based on customers’ feedback.

Follow the steps below to identify content that falls into above category by applying multiple layers of filtering.

Step 1. To start exploring your data, go to the SKyEyeUserComments portal

  • Type a service name of your choice into the Search Field and Press Enter or click the Magnifying Glass button. NOTE: we will use “Active Directory” as an example

Step 2. Apply a “CSAT” filter to narrow down the content with CSAT less than 60%.

  • On the left-hand menu expand CSAT% filters and select “less than 60%”

Step 3. Apply “Visitors Count” filter to narrow down the content with a visitor count of 5000 or higher.

  • Expand “Visitors Count” and select “5000 and more”.

Step 4. Apply a “Category Confidence” filter to narrow down the content in a feedback category with a high confidence level of 75% or higher.

  • Expand “Category Confidence” and select “75 or more”.

Step 5. Apply a “Comment Category” filter to narrow down the content in the specific feedback category of your choice.

  • On the left-hand menu expand “Comment Category” then click on the sub menu to select category of your choice. In this example, we will select “Content Incomplete” category.

Step 6. Export final dataset to CSV

  • On the right side of the portal click “Export to CSV” link.
  • In the popup properties form select all properties you wish to include in the export data file.
  • In the CSV file sort (or filter) by Article Title to analyze customers’ feedback.

The overall CSAT ratings for a service can be impacted by low CSAT of localization content. To find out if it’s the case with the service you are investigating, follow the steps below.

Step 1. Explore the effect of Loc content ratings on CSAT

  • Navigate to the SkyEye Analytics Hub http://skyeyereports.cloudapp.net/ , In “Executive Summaries” section, open the Scorecard report.
  • To compare EN to Loc CSAT for a four weeks period, select a date the week is ending in.
    NOTE: we will use four weeks’ period ending on 01/14/2017 to investigate API Apps as an example
  • Under Azure App Services review ‘%Helpful Ratings” for “Hero & Get Started” articles For “API Apps” (Locale =”all”)
  • In the Locale filter change to “en-*” and review the value in the ‘%Helpful Ratings’ column for the API Apps
  • If you observe a drop in CSAT ratings for “all” locals compare to “en-*”, the overall CSAT is impacted by low ratings of localized content.

Step 2. Explore CSAT trend for localized content in comparison to EN content

  • Navigate to the SkyEye Analytics Hub http://skyeyereports.cloudapp.net/ , open the Topics (Services) report, and click the “Services: Trend” tab.
  • Select a service of your choice from the service list and Tier1 (tier one includes loc languages) for a “Tier” list.
  • Observe trends for selected service and loc languages.
  • Identify the dates when dropped in ratings occurred.

Step 3. Explore customer comments and ratings for loc content

  • Navigate to the “Services: Topic Drill Down” tab.
  • Select a service of your choice and select “Tier1” from a Tier list.
  • In the Summaries per topic table sort on “NotHelpful” ratings in descending order.
  • Review the Comments section to analyze customers’ comments.
  • Did the customer comments indicated a localization lag or a product related issue?