Developer Analytics with Application Insights and HockeyApp

Many projects operate a rapid DevOps cycle. They build and distribute their applications, get feedback about how it performs and what users do with it, and then use that knowledge to plan further development cycles.

To monitor usage and performance, it's important to have telemetry from the live application, as well as feedback from the users themselves.

Many systems are built from multiple components: a web service, backend processors or data stores, and client software running either in the user's browser or as an app in a phone or other device. The telemetry from these different components has to be taken together.

Some releases have restricted distribution to designated testers; we have also organized flighting (tests of new features with restricted audiences), and A|B testing (parallel tests of alternative UI).

Managing distributions and integrating monitoring over multiple client and server components isn’t a trivial task. This process is an essential part of the architecture of the application: we can’t create a system of this kind without an iterative development cycle and good monitoring tools.

In this article, we'll look at how the monitoring aspects of the devOps cycle fit in with the other parts of the process.

If you'd like to look at a specific example, there's an interesting case study that has multiple client and server components.

A DevOps cycle

Visual Studio and Developer Analytics tools offer a well-integrated devOps experience. For example, here is a typical cycle for a web application (which could be Java, Node.js, or ASP.NET):

Web app devops cycle

Device and desktop apps

For device and desktop apps, the distribution part of the cycle is slightly different, because we aren't just uploading to one or two servers. Instead, a successful build and unit test can trigger uploading to HockeyApp. HockeyApp supervises distribution to your team of test users (or to the general public, if you prefer).

Device devops cycle

HockeyApp also collects performance and usage data, in the forms of:

  • Verbatim user feedback with screenshots
  • Crash reports
  • Custom telemetry coded by you.

Once again, the devOps cycle is completed as you make your future development plans in the light of the feedback gained.

Setting up Developer Analytics

For each component of your application - mobile or web or desktop - the steps are basically the same. For many types of app, Visual Studio automatically performs some of these steps.

  1. Add the appropriate SDK to your app. For device apps, it's HockeyApp, and for web services it's Application Insights. Each has several variants for different platforms. (It's also possible to use either SDK for desktop apps, though we recommend HockeyApp.)
  2. Register your app with the Application Insights or HockeyApp portal, depending on the SDK you used. This is where you'll see analytics from your live app. You get an instrumentation key or ID that you configure into your app so that the SDK knows where to send its telemetry.
  3. Add custom code (if desired) to log events or metrics, to help you with diagnostics or to analyze performance or usage. There's a lot of monitoring built in, so you won't need this on your first cycle.
  4. For device apps:
    • Upload a debug build to HockeyApp. From there you can distribute it to a team of test users. Whenever you upload subsequent builds, the team will be notified.
    • When you set up your continuous build service, create a release definition that uses the plug-in step to upload to HockeyApp.

Analytics and Export for HockeyApp telemetry

You can investigate HockeyApp custom and log telemetry using the Analytics and Continuous Export features of Application Insights by setting up a bridge.

Next steps

Here are the detailed instructions for different types of app: