Viewing content related data in SkyEye reports and search

SkyEye provides a variety of ways to view and make sense of content-related data for Docs and other key sites. This page provides an overview of SkyEye data categories and reports (including search capabilities).

Data categories in SkyEye

This section provides a brief overview of data categories in SkyEye. (For a summary of common metrics, see Quick references and job aids.)

Usage (page views, visitors count, time in page)

The usage metrics are indicative of the adoption of our content. Typically those are aggregated by day, week, month. Some very popular topics have in excess of 5,000 to 10,000 page views and users per month, the average topics has 100-500 page views per month while others (even important ones) could have down to 5-10 users per month. The majority of the reports contain columns or charts with the page views and visitors per week or per month. If you want to look only at usage the report called "Referer Location Browser OS" on the SkyEye home page provides charts and filters only for usage. It is possible to see in SkyEye also each individual page view. For this use the following:

  • Look for the "Master" or "Topics" report under the "Data Exploration" section on the home page. On this one after you get the data click on any of the numbers there and you can drill down to the page view.it is possible to drill down to individual page views and view the click paths of single visitors.
  • The concept of "visit" has not yet been implemented in SkyEye – for this reason, data like "visits", "bounce rate", and "entry/exit pages" are not provided. More visit-level data will be available in the future.

Ratings, verbatims, and comments

The ratings and verbatims are coming from the "Is this page helpful?" control with a blue yes and no button that shows up on the bottom right corner of any page on docs.microsoft.com. Total ratings represent the number of users who click either "yes" or "no". The CSAT or % helpful is the number of click on "yes" divided by the total of clicks on "yes" or "no". When the users click "No" they are also presented with a dialog box where they can enter anonymous feedback. This is shown as "verbatim" in the respective reports. On the SkyEye home page look for a report called "Verbatims" under the "For writers" section and those shows all the verbatims filterable by author, article, product, service and other attributes. Here are a few key points:

  • Content ratings are used to derive the "% Helpful Ratings" metrics in the C+E content scorecard.
  • Verbatim feedback is classified using a machine learning algorithm – this helps content owners filter the data. See the SkyEye Verbatims report for an example. (If the category looks incorrect, click the link in the CategoryFix column and select the best category. This will help improve the model over time.)
  • Reports are also available on LiveFyre comments (for Docs), Disqus comments, and verbatim product feedback (for a subset of products and services).

Contributions

The Pull request report enables content owners to track customer contributions to Docs pages. This report is also filterable to only show open pull requests from end user awaiting your approval.

Content quality data

Other data in SkyEye relates specifically to content quality. Here are a few examples:

  • The content freshness date indicates the last time that the content owner reviewed the page and asserted that it is up to date matching the shipped product or service. The ms.date attribute is used to set the freshness date in GitHub. This date is also visible to the end user right under the article type. To see at a glance the freshness of your articles user the SkyEye Freshness PowerBI report where you can filter by author, product, service and many other filters and see the freshness for the articles selected.
  • It is possible to find pages with broken links by applying a filter in SkyEye Search.
  • An Acrolinx report provides data on content quality from an Acrolinx perspective.

International data

International data is integrated into most SkyEye reports. (It will be added to SkyEye search in the near future.)
SkyEye provides autotranslated versions of international verbatim feedback and comments.

And: Metadata

Content metadata is an important part of the SkyEye foundation. Provided by the content owner (product/service, author, "freshness date") and the authoring/publishing system (repo, language), metadata enables SkyEye to filter reports, queries, and action items.
Metadata requirements are covered in depth in the metadata section of this contributor guide. Here are a few key points about metadata:

  • The content owner is responsible for adding required metadata to OPS pages. Note there is an approved "whitelist" of values for the scorecard and other key reports.
  • SkyEye MetaCop can be used to manually validate metadata in markdown files. Just enter in there the Github URL path to a .md file or copy/paste the meta block from the .md file and the tool will tell you if there is any missing or incorrect metaadata. The report on http://aka.ms/skyeye/meta/bad allows you to get a list of articles with bad ot missing meta for a given author, product, service etc. Note that work is underway to build an OPS metadata validation service. When completed this will enable content owners to validate metadata in Visual Studio Code, and will also validate it at publishing/build time.
  • Metadata values are refreshed in SkyEye on a weekly basis so if you update values in your article it will reflect in reports the next Monday.
  • It is always best to update metadata in the original source file. If there is no way to update the original source file, or an "emergency" update is needed before the weekly metadata refresh, it is possible to directly update the metadata in SkyEye. To do so, find the page in the SkyEye Search by entering in there the article title or URL, click on the link called "(Meta)" next to that article and update the metadata. Caution: take care when making direct metadata updates in SkyEye abd esure the values you enter are accurate.
  • In some cases where metadata is missing, SkyEye uses a "self-healing" process to guess the correct values. In case of a conflict, the values in the markdown files will override the "self-healed" values.
  • If you need to add a new metadata value to the "whitelist" email skyeye@microsoft.com.
  • If your metadata is incorrect or incomplete the page views data is not lost, but it will get bucketed under the "unknown" category and will make it hard to filter and will make the highher level reports less accurate.

SkyEye reports and search

This section provides a brief overview of SkyEye search, reports, and scorecards.

SkyEye Search is a very convenient way to access SkyEye data. Here are a few examples.

Author summary

The author summary includes a variety of information about the content owned by a single author. To access this summary, search on the name or alias of a content owner. (If the report does not appear, click Authors in the left navigation and select the alias.)

SkyEye search: Author summary

Service summary

The service summary includes a variety of information about the content for a single product or service. To access this summary, search on the name or alias of a content owner. (If the report does not appear, click Products or Services on the left side of the page and select the alias.)

SkyEye search: Service summary

Page-level data

To see data on a specific page, search on the title or URL of the page.

SkyEye search: Single page

Filters

To filter search results, click items in the list on the left side of the page.

SkyEye search: Search filters

Reports

SkyEye provides a number of reports (to see the list, go to the SkyEye home page). Here are just a few examples.

Topics (Services) report

The Topics (Services) report provides a combined view of page views, ratings, and verbatim feedback. The report is filterable by service, language, author, and topic type – and it is possible to export the lists of pages and feedback items to Excel for further analysis.

SkyEye report: Topics (Services)

Reports for verbatims and comments

The Verbatims, LiveFyre, and Product Feedback reports all provide views into verbatim comments provided by customers.

Data exploration

The SkyEye MasterRollup, TopicRollup, InstancesView, and SessionView reports enable data exploration down to the single page view and visitor. Here is one way to go about it:

  • In the MasterRollup report, select a product or service (at daily, weekly, or monthly granularity). Click one of the links in the Articles w Page Views column to open the TopicRollup report.
  • In the TopicRollup report, identify the topic of interest, and click the link in the PageView Count column to open the InstancesReview report.
  • In the InstancesView report, identify the specific page view of interest, and click the link in the VisitorId column to open the SessionView report.
  • In the SessionView report, browse the sequence of pages visited by that single visitor.

Here is an example of a user working with Git, R Client, R Server, and Linux VMs in a single day:

SkyEye report: Data exploration

Content scorecard

The C+E content scorecard was developed to provide C+E executives with a view of the content business. The different fields are described in the MeasuresDefn tab of the scorecard. (There is also an international scorecard with more detail about international content.)