Configuring Azure Diagnostics Performance Counter Monitoring in SCOM 2007 R2
In my last post, I provided a primer on installing SCOM 2007 R2 along with the Azure Management Pack (which is still currently in Release Candidate status). In the post before that, I showed you how to setup the performance counters in your web and worker roles in Visual Studio 2010 so you could generate the diagnostics that SCOM will ultimately monitor. In this post, I want to take the final step of showing you how to properly configure SCOM 2007 R2 so you can monitor the performance counters generated by Azure diagnostics in the Operations Console. This blog is designed to augment the existing SCOM 2007 R2 Azure Management Pack documentation, which can be found here on the same page as the download. The documentation is pretty good, but it is not scenario-based so it may not be clear exactly where you need to go or what you have to do to get your counters setup properly. I will walk you through the entire process and hopefully provide useful commentary on why you’re doing what you’re doing. A special shout out goes out to my colleague and resident SoCal SCOM expert Brian Wren, who coached me on what you are reading in this post.
If you haven’t already, please be sure to install the System Center Operations Manager 2007 R2 Authoring Resource Kit, which you will need to create the performance collection rules that will encapsulate your performance counters. You must use the Authoring Console installed by the Resource Kit to create rules, for reasons you will see later. It is important for me to point out that the approach here begins in the Operations Console, though the creation of the rules is typically authored first in the Authoring Console, as documented in Azure Management Pack Guide. One benefit of creating in the Operations Console is that the process is a bit easier if you’re just starting out, as it automates some steps you would have to perform manually in the Authoring Console. One thing you must remember if you want to use the Authoring Console with CU4 is to make note of the known issue regarding exporting of management packs from the Operations Console. Just search on "New management pack edit issue" in the CU4 documentation. Basically, the updated reference is not recognized by the Authoring Console, so you will have to set the SystemCenter reference in your exported management pack to 6.1.7221.0, or otherwise you will get a bad reference message upon import with the Authoring Console. I know, really dumb bug.
Even though you have already installed the Windows Azure Management Pack, which has its own set of “out of the box” rules, you will want to create your own management pack instead of modifying the existing pack, since we will be modifying this and making it specific to your desired counters. You will find the installed pack in the list of management packs when you select the Management Packs node in the Administration view. So let’s go ahead and create your management pack here, as seen below, by right-clicking the Management Packs node and selecting the Create Management Pack menu item.
Give the management pack a name and select Create on the next page in the wizard, as seen below.
Now that we have the management pack created, we can add our counters to it. Let’s now select the Authoring view and select the Rules node as seen below. Here you will see an enormous list of rules from a number of management packs, but we will scope this down to make it more manageable for our purposes, related to Windows Azure.
Now select the Scope button from the toolbar, which will display the Scope Management Pack Objects dialog box. Select the View all targets radio button, and scroll down until you see the targets that begin with Windows Azure. For our scope, we will select only the following targets:
- Windows Azure Deployment
- Windows Azure Hosted Service
- Windows Azure Role
- Windows Azure Role Instance
Thus, your view should look like the following below:
We will only use the Windows Azure Role Instance in this post, but this list will give you an idea of what can be monitored at the Hosted Service, Deployment, Role, and Role Instance levels. Below you can see the end result of the scoping, which includes performance counters I already created for my existing SCOM configuration. (The reason you don’t see any for Deployment in this scope is because there are no “out of the box” rules for Deployment.)
Okay, so now let’s begin the process of creating a performance collection rule. Right click on the Rules node and select the Create a new rule… menu item. From here you will see the Create Rule Wizard dialog. Under the Collection Rules node, you must select the Performance Based node and then the Windows Performance node. Next, in the Management pack section, select the management pack you created previously as your destination management pack. Your dialog should look similar to the screenshot below.
Select the Next button, which will bring you to the Rule Name and Description page of the wizard. In this dialog, provide a rule name and description. Select Performance Collection as your rule category, as seen below.
You must next select the rule target on this page, so now select the Select… button. On this dialog, you want to select the View all targets radio button, and then scroll down to the Windows Azure Role Instance item in the list. You can select this item, as seen below, and then select the OK button, but this will collect data for all role instances across hosted services you are monitoring. You will probably only want to collect data for instances within a particular hosted service, so in that case you would select the Windows Azure Role Instance (<your hosted service name>) item in the list (as seen in the screenshot below, which in this case is loggingservice).
Once you have selected your Windows Azure target items to be monitored, you can next select the specific counter that you desire to monitor in the Performance Object, Counter, and Instance page of the wizard. The easiest thing to do is to simply choose the Select… button. From here you will be presented with a dialog that will allow you to select your desired performance counter, as seen below. I typically set the interval for collection of the counters at 5 minutes. In the next page titled Optimized Performance Collection Settings, accept the default setting that will not use optimization.
Continue this process for all of the desired performance counters. You will now have a list of rules that look similar to the screenshot below.
Now the fun begins, which is critical to your success at capturing the diagnostics for any given counter. Even though you have created your rule, you must now configure it both here in the SCOM Operations console as well as the Authoring Console. Let’s now begin rule configuration in the Operations Console by double-clicking on a rule to bring up the Properties dialog. Select the Configuration tab, which will look as seen below.
The problem is that the data source, by default, is not pointing to the correct location of your diagnostics data. So we have to correct this issue. And that is where the Authoring Console comes in, which is where we must go to configure each performance collection rule for Windows Azure. We will begin this process by heading to the Administration view, selecting the desired management pack, and exporting it to disk for use in the Authoring Console, as seen below. The file will be in the .xml format.
Now launch the Authoring Console, which you should find added after installing the Authoring Resource Kit in the System Center Operations Manager 2007 R2 program group. Before you can us this, you will have to copy certain management packs the Authoring Console must reference in order to run to the Authoring Console installation folder. Please see the Creating a Performance Collection Rule for Windows Azure section of the Windows Azure Management Pack documentation for more details, or when prompted by the Authoring Console, you can just search for the required management pack in the install folders for SCOM 2007 or the Windows Azure Management Pack.
Once you have the required references resolved, open the management pack you just exported. Click the Health Model view, and then select the Rules node in the navigation pane. Your Authoring Console should look similar to the screenshot below. Note that there will be an entry for each of your corresponding rules, but the internal name will be a nasty GUID prefixed with “MomUIGeneratedRule.” This isn’t a big problem, because you’re only using this console to configure the data source for each of your performance collection rules, and the Details pane below shows you the Name and Description you have given the rule.
Let’s now open a rule and begin the configuration process. Double-click on a rule, and you should see a dialog similar to the screenshot below. Note that the name of your rule is still intact, it just has the nasty looking, prepended GUID as a unique identifier for the rule.
Now click the Modules tab, where you will see the Data Sources section that has to be modified, as seen below.
What you will do here is to delete the existing data source entry, and then replace it with a new entry by selecting the Create… button, which will bring up a dialog as seen below. But don’t delete the existing just yet, as you will need to copy the CounterName and ObjectName values (amongst others) to paste into the new entry, as seen below. So now would be a good time to open notepad and copy the values you desire. When you’re done, select the OK button.
Now select the Create… button, which will display the dialog seen below. Type “azure” in the Look for: textbox to narrow down the list, and select the Microsoft.SystemCenter.Azure.RoleInstance.PerformanceCounter.CollectData.DS type entry. Then enter a Module ID with any name you would like to give it (e.g., “AzureDS”, which I used for all of my rules, since the scope of the ID is only within the rule itself). Your dialog should look similar to below.
Now we must edit the data source we just created to adjust values, which will look like this initially.
Now we will begin setting values, as seen below.
These are not all of the values, necessarily, but minimal values provided by default. For this particular counter, we need to add the InstanceName and AllInstances values, which means we must select the Edit… button and add these manually.
After saving and closing the XML you edited in notepad, you will see the following below. Don’t worry about the Run as profile: dropdown, as the Windows Azure Run As Profile Proxy will be configured on export. You can now select the OK button, and when you’re taken back to the previous dialog, just delete the old data source and select the OK button.
Now that you are done configuring the performance collection rules, you must now export these rules back to SCOM 2007. First, select the Save button to save the modified management pack to disk. Next, select the Tools menu, and then select the Export MP to Management Group… menu item.
Choose your server name and select the Connect button, as seen below.
Now let’s go back to the Operations Console, where we should have the Rules node selected in the Authoring view. If this node was already selected, then refresh the page. Double-click on any of the rules you modified in the Authoring Console, go to the Configuration tab, and you should see something similar to the following (dialog has been intentionally resized so you can see all of the text of the data source). Your data source and Run as profile are now configured.
Whew! You have now properly configured SCOM 2007 to capture your performance collection data. Please feel free to leave a comment if any questions or concerns.