How to Get Diagnostics Info for Azure/PHP Applications–Part 1

  • Article

June 7, 2012 update: The Microsoft Windows Azure team has released a new Windows Azure SDK for PHP. This release is part of an effort to keep PHP client libraries up to date with new Windows Azure features and to make PHP a first-class citizen in Windows Azure. The latest client libraries are on GitHub: https://github.com/WindowsAzure/azure-sdk-for-php. While the SDK hosted on CodePlex will continue to work for the foreseeable future, it is strongly recommended that new PHP/Windows Azure application use the SDK hosted on GitHub.

The work done by Maarten Balliauw and other contributors in building the SDK hosted on CodePlex was critical in unifying the PHP developer experience for Windows Azure. The Windows Azure team is grateful to these contributors for their pioneering work and looks forward to their continued support (and yours!) in adding to the new SDK on GitHub.

Thanks,

      
The Windows Azure Team

In this post, I’ll look at why getting diagnostic data for applications in the cloud is important for all applications (not just PHP), provide a short overview of what Windows Azure Diagnostics is, and show how to get diagnostics data for PHP applications (although much of what I look at is applicable regardless of language). In this post I’ll focus on how to use a configuration file (the diagnostics.wadcfg file) to get diagnostics, while in Part 2 I’ll look at how to to this programmatically.

Why get diagnostic data?

There are lots of reasons to collect and analyze data about applications running in the cloud (or any application, for that matter). One obvious reason is to help with troubleshooting. Without data about your application, it’s very difficult to figure out why something is broken when it breaks. However, gathering diagnostic data for applications running in the cloud takes on added importance. Without diagnostic data, it becomes difficult (perhaps impossible) to take advantage of scalability, a basic value proposition of the could. In order to know when to add or subtract instances from a deployment, it is essential to know how your application is performing. This is what Windows Azure Diagnostics allows you to do. (Look for more posts soon about how to scale an application based on diagnostics.)

What is Windows Azure Diagnostics?

In understanding how to get diagnostic data for my PHP applications running in Azure, it first helped me to understand what this thing called “Windows Azure Diagnostics” is. So, here’s the 30,000-foot view that, hopefully, will provide context for the how-to section that follows…

Windows Azure Diagnostics is essentially an Azure module that monitors the performance (in the broad sense) of role instances. When a role imports the Diagnostics module  (which is specified in the ServiceDefinition.csdef file for a role), the module does two things:

  1. The Diagnostics module creates a configuration file for each role instance and stores these files in your Blob storage (in a container called wad-control-container). Each file contains information (for example) about what diagnostic information should be gathered, how often it should be gathered, and how often it should be transferred to your Table storage. You can specify these settings (and others) in a diagnostics.wadcfg file that is part of your deployment (the Diagnostics module will look for this file when it starts). Regardless of whether you include the diagnostics.wadcfg file with your deployment, you can create or make changes programmatically after deployment (the Diagnostics module will create and/or update the configuration files that are in Blob storage).
  2. According to settings in the configuration files in Blob storage, the Diagnostics module begins writing  data to local storage and transferring it to Table storage (to enable the transfer you have to supply storage connection string information in your ServiceConfiguration.cscfg file).

So that’s the high-level view (import a module, configure via a file or programmatically, diagnostics info is written to your Azure storage account). Now for the details…

How to get diagnostic data using a configuration file

There are two basic ways you can get diagnostics data for a Windows Azure application: using a configuration file and programmatically. In this post, I’ll examine how to use a configuration file (I’ll look at how to get diagnostics programmatically in Part 2). I’ll assume you have followed this tutorial, Build and Deploy a Windows Azure PHP Application, and have a ready-to-package PHP application.

Note: Although I will walk through this configuration in the context of a PHP application, the configuration steps are the same for any Azure deployment, regardless of language.

Step 1: Import the Diagnostics Module

To specify that a role should import the Diagnostics module, you need to edit your ServiceDefinition.csdef file. After you run the default scaffolder in the Windows Azure SDK for PHP (details in the tutorial link above), you will have a skeleton Azure PHP application with a directory structure like this:

image

Open the ServiceDefinition.cscfg file in your favorite XML editor and make sure the <Imports> element has this child element:

 <Import moduleName="Diagnostics"/>

That all that’s necessary to import the Diagnostics module.

Step 2: Enable transfer to Azure storage

To allow the Diagnostics module to transfer data from local storage to your Azure storage account, you need to provide your storage account name and key in the ServiceConfiguration.cscfg file. Again in your favorite XML editor, open this file and make sure the <ConfigurationSettings> element has the following child element::

 <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" 
          value="DefaultEndpointsProtocol=https;AccountName=your_account_name;AccountKey=your_account_key"/>

Obviously, you’ll have to fill in your storage account name and key.

Step 3: Edit the configuration file (diagnostics.wadcfg)

Now you can specify what diagnostic information you’d like to collect. Notice that included in the directory structure shown above is the diagnostics.wadcfg file. Open this file in an XML editor and you’ll begin to see exactly what you can configure. I’ll point out some of the important pieces and provide one example.

In the root element (<DiagnosticMonitorConfiguration>), you can configure two settings with the configurationChangePollInterval and overallQuotaInMB attributes:

  1. The frequency with which the Diagnostic Monitor looks at the configuration files in your blob storage for updates (see the What is Windows Azure Diagnostics section above for more information) is controlled by the configurationChangePollInterval.
  2. The amount of local storage set aside for storing diagnostic information is controlled by the overallQuotaInMB attribute. When this quota is reached, old entries are deleted to make room for new ones.

In the example here, these settings are 1 minute and 4GB respectively:

 <DiagnosticMonitorConfiguration xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration" 
     configurationChangePollInterval="PT1M" 
     overallQuotaInMB="4096">

Note: if you want to increase the value of the overallQuotaInMB setting, you may also need to edit your ServiceDefinition.csdef file prior to deployment – details here.

The following table describes what each of the child elements in the default diagnostics.wadcfg file does. (For more detailed information, see Windows Azure Diagnostics Configuration Schema.)

Element

Description

<DiagnosticInfrastructureLogs>

Logs information about the diagnostic infrastructure, the RemoteAccess module, and the RemoteForwarder module.

<Directories>

Defines the buffer configuration for file-based logs that you can define.

<Logs>

Defines the buffer configuration for basic Windows Azure logs.

<PerformanceCounters>

Logs information about how well the operating system, application, or driver is performing. For a list of performance counters that you can collect, see List of Performance Counters for Windows Azure Web Roles.

<WindowsEventLog>

Logs events that are typically used for troubleshooting application and driver software.

The bufferQuotaInMB attribute on these elements controls how much local storage is set set aside for each diagnostic (the sum of which cannot exceed the value of the overallQuotaInMB attribute). If the value is set to zero, no cap is set for the particular diagnostic, but the total collected at any one time will not exceed the overall quota.

The scheduledTransferPeriod attribute controls the interval at which information is transferred to your Azure storage account.

As an example, this simple configuration file collects information about specified performance counters (the sampleRate attribute specifies how often the counter should be collected):

 <DiagnosticMonitorConfiguration xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration" 
     configurationChangePollInterval="PT1M" 
     overallQuotaInMB="4096">
    <PerformanceCounters bufferQuotaInMB="0" scheduledTransferPeriod="PT5M">
       <PerformanceCounterConfiguration 
          counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT1M" />
       <PerformanceCounterConfiguration 
          counterSpecifier="\Memory\Available Mbytes" sampleRate="PT1M" />
       <PerformanceCounterConfiguration 
          counterSpecifier="\TCPv4\Connections Established" sampleRate="PT1M" />
    </PerformanceCounters>
   </DiagnosticMonitorConfiguration>
Step 4: Package and deploy your project

Now, you are ready to package and deploy your project. Instructions for doing so are here: Build and Deploy a Windows Azure PHP Application.

Step 5: Collect diagnostic information

After you deploy your application to Windows Azure, the Diagnostics module will begin writing data to your storage account (it may take several minutes before you see the first entry). In the case of performance counters (as shown in the example configuration file above), diagnostic information is written to a table called WADPerformanceCountersTable in Table storage. To get this information, you can query the table using the Windows Azure SDK for PHP like this:

 define("STORAGE_ACCOUNT_NAME", "Your_storage_account_name");
 define("STORAGE_ACCOUNT_KEY", "Your_storage_account_key");
 $table = new Microsoft_WindowsAzure_Storage_Table('table.core.windows.net', STORAGE_ACCOUNT_NAME, STORAGE_ACCOUNT_KEY);
    
 $metrics = $table->retrieveEntities('WADPerformanceCountersTable');
  
 $i = 1;
 foreach($metrics AS $m) {   
     echo $i.". ".$m->DeploymentId . " - " . $m->RoleInstance . " - " . $m->CounterName . ": " . $m->CounterValue . "<br/>"; 
     $i++;
 }

Of course, getting this data can be somewhat trickier if you have several roles writing data to your storage account. And, this begs the question, what do I do with this data now that I’ve got it? So, it looks like I have plenty to cover in future posts.

Thanks.

-Brian

Share this on Twitter