January 2015
Volume 30 Number 1
Microsoft Azure - Cross-Platform Cloud Automation with JavaScript
By Steven Edouard | January 2015
Not everyoneknows Microsoft Azure has the Cross-Platform Command-Line Interface (xplat-cli) for managing Azure Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS) services, but it does. For years, Microsoft has advocated using Windows PowerShell as a scripting language, which worked well for provisioning cloud-based resources. But now that more developers are involved in deploying and moving applications to the cloud, it makes more sense to use JavaScript. The Azure Service Management REST APIs make this cross-platform implementation easier. The official Azure xplat-cli is implemented on top of Node.js, the same runtime often used to build high-performance Web sites and APIs with JavaScript.
Building on top of Node.js makes scripting truly cross-platform. It means you can run your scripts for managing virtual machines (VMs), SQL Databases, Web sites and virtual networks from any machine. This frees you to do automation scripts in a wider variety of languages and for a wider variety of client computers (see Figure 1).
.png "Supported Scripting Environments")
Figure 1 Supported Scripting Environments
There are many advantages to scripting cloud resource provisioning. For example, the Web portal isn’t repeatable and requires human input. Another advantage is being able to source control the steps to provision and deploy your service using different specific configuration values. In this article, I’ll walk through scripting the process of creating VMs associated with an Affinity Group and a virtual network using Node.js as our script automation environment.
The Cross-Platform CLI
Like the Azure PowerShell CLI, the cross-platform version offers powerful script ability to manage IaaS and PaaS services. This lets you describe your infrastructure with code, share it with your team via source control, and automate the creation of development, test, and production application deployments.
To get started with the xplat-cli, head over to Nodejs.org and install the appropriate Node.js package for your system. After installing Node, you can view the azure-cli package on npmjs.org/package/azure-cli.
The package listing gives you all the relevant documentation on the CLI, as well as the simple command line to install the package. This package is a CLI, so it makes sense to install it in the global Node.js package store by specifying –g, so you can make Azure command-line calls from any directory, instead of having to explicitly invoke the CLI from a specific directory:
npm install azure-cli -g
After a few lines of output from the Node Package Manager, you’ll have your azure-cli installed and you can start using it immediately.
Completely Self-Discoverable CLI
The great thing about the CLI is that it’s completely self-discoverable. You can easily get started just by typing commands you’d like to know more about. Figure 2 shows what happens when you call the base CLI command Azure.
.png "The Azure xplat-cli")
Figure 2 The Azure xplat-cli
There’s a plethora of possible commands you can do from the base Azure command. Say you notice the vm command from the help output in Figure 2. You can drill into the command to better understand how to use it by just typing the azure vm command (see Figure 3):
.png "CLI Command to Create a Virtual Machine")
Figure 3 CLI Command to Create a Virtual Machine
This same pattern applies to all commands in the CLI. It creates an intuitive interface for exploring the command line.
Access Your Subscription from the CLI
Not surprisingly, most of the Azure commands won’t actually work until you sign up for an Azure subscription and tell the command-line tool what Azure subscription to use. Similar to the Windows PowerShell tools, you need to give the CLI access to your subscription. There are two ways to do this—via an Active Directory account using a Microsoft Organizational Account with the login command or with a portable .publishsettings file using the account import command. The latter method is more portable as it doesn’t require the Organizational Account setup. There’s an easy way to download your .publishsettings by executing:
azure account download
This will actually open your default browser to a link that will have you sign in to your Azure account and download your .publishsettings file. After the file downloads, you can use the account import command to load up the file:
azure account import <PATH TO .publishsettings>
This will set up the command line to use the Azure subscription specified in the .publishsettings for any Azure command called on the CLI. To see if the CLI works, call the site list command to output all Web sites running on your current subscription.
Create a Dev VM from the CLI
VMs are particularly useful for DevOps scenarios. You can easily provision them through the xplat-cli.
I’ll eventually use the CLI to automate these tasks by scripting them with JavaScript on Nodejs. For example, I’ll create a new Ubuntu VM. To do so, I need to find a VM image. I can fetch a list of available VMs by executing the command:
azure vm image list
The command output shows a list of available public and private images you can use to provision a VM. The public images are curated by Microsoft, but you can use your own uploaded images, as well.
To create a VM, you need a datacenter location. To get a list of valid locations where you can place your VM, execute this command:
azure vm location list
You can use this location as input to the vm create command to create a new VM:
azure vm create <yourdomainname> <imagename> <username> <password> --location '<selected datacenter region>'
Now that you’ve successfully created a VM from the command line, take a look at how easily you can do this using a script.
Script Away Azure Resources in JavaScript
More developers are trying to use JavaScript for just about everything. Provisioning resources in the cloud is no exception. This supports other scripting languages, as well, such as Bash, Batch and even Windows PowerShell. The choice of language boils down to the platforms on which you’ll run and your language preferences.
Microsoft will soon release the Azure Resource Manager, which will use a more declarative model instead of the programmatic model. Until Azure Resource Manager is ready, you can use some of these techniques to automate your provisioning to the cloud.
If you browse closely on the azure-cli npm documentation (npmjs.org/package/azure-cli), you’ll notice that there’s a library dependent on azure-cli called azure-scripty. It turns out someone in the open source Node.js community has already decided to write a command-line adapter for the azure-cli for JavaScript. Because this library is a command-line wrapper, you can use all the same commands as on the command line in JavaScript. Here’s an example that will list all of our VMs:
// Get the list of existing VMs on this subscription
scripty.invoke('vm list', function (err, result) {
if (err) {
return cb(err);
}
for (var i in result) {
console.log(result[i].VMName);
}
});
To get started, take a look at how you can make a batch of VMs. First, install the necessary packages:
# a command-line wrapper for the azure-cli
npm install azure-scripty
# a small library that helps manage asynchronous tasks
npm install async
# a library that makes using JSON configuration files easy
npm install nconf
To avoid hardcoding the configuration settings for staging, test and production, use a configuration file to keep the scripts flexible enough for different environments such as development, test and production. Call this configuration file config.json (see Figure 4).
Figure 4 Config.json Maintains Flexible Settings
{
"vnet_name":"azuretedvnet2",
"vm_count": "3",
"vm_image_name":
"b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04-LTS-amd64-server-20140416.1-en-us-30GB",
"vm_base_name":"sedouardmachine",
"vm_username":"sedouard",
"vm_password":"p@assW0rd",
"affinity_group":{
"name":"azureteddemoaffinitygroup",
"label":"sedouard",
"location":"West US"
}
}
You can use that configuration file to set up a developer environment, complete with three Ubuntu VMs that are part of the affinity group (azureteddemoaffinitygroup) and a corresponding virtual network (azuretedvnet2). This approach is declarative, meaning you only need to make changes to a configuration file to change the way your development environments are set up. You can also have additional versions for staging, test and production—and source control each.
The custom JavaScript code will parse the configuration file so the dependencies, such as affinity groups, will be provisioned with the correct names and values. Inside the config.json file, the script can read the desired name for the affinity group (azureteddemoaffinitygroup). From there, it can query check to see if the affinity group has already been created by looping through the list of provisioned affinity groups that already exist in the datacenter (see Figure 5).
Figure 5 Create an Affinity Group
//////////////////////////////////////////////////////////////////////////////
//Creates an affinity group if the specified one doesn't exist
//////////////////////////////////////////////////////////////////////////////
function createAffinityGroup(cb) {
console.log('Getting affinity groups...');
scripty.invoke('account affinity-group list', function (err, result) {
// Check for errors.
if (err) {
return cb(err);
}
console.log('Current affinity groups');
// For debugging purposes, list out names of existing affinity groups.
for (var i in result) {
console.log(result[i].name);
}
// Get the name of the desired affinity group from the config.json file.
var affinityGroup = nconf.get('affinity_group').name;
var label = nconf.get('affinity_group').label;
// Verify that affinity group hasn't already been created.
for (var i in result) {
if (result[i].name === affinityGroup && result[i].label === label) {
// The affinty group to use in the config already exists.
return cb();
}
}
// Create affinity group because it doesn't exist.
console.log('Specified affinity group ' + affinityGroup +
' does not exist, creating new one...');
var cmd = {
command: 'account affinity-group create',
positional: [affinityGroup],
location: '\"West US\"',
label: label
};
scripty.invoke(cmd, function (err) {
if (err) {
cb(err);
}
return cb();
});
});
}
Notice how the azure-scripty library lets you easily parse output from the azure-cli via callback function result argument. It also lets you create CLI commands by declaratively specifying the command, positional and named parameters.
The pattern in Figure 6 is the same one you use to create your virtual network and uses the azure-scripty library in the same way.
Figure 6 Create a Virtual Network
//////////////////////////////////////////////////////////////////////////////
//Creates the config specified vnet, if it doesn't already exist
//////////////////////////////////////////////////////////////////////////////
function createVirtualNetwork(cb) {
console.log('Getting networks...');
scripty.invoke('network vnet list', function (err, result) {
if (err) {
return cb(err);
}
var networkName = nconf.get('vnet_name');
for (var i in result) {
console.log(result[i].name);
}
// Check if VNet name listed in config.json exists.
// If it doesn't, create the VNet.
});
}
Some of the code in Figure 6 has been omitted for brevity, but the same pattern exists. The needed resource is specified in the configuration file. You’ll need to provision it if it doesn’t exist.
You can create a batch of VMs once you’ve created the dependencies (the affinity groups and networks). The number of VMs you’ll create is specified in the vm_count field in the config.json configuration file. Following the same pattern, first list the current VMs on the subscription, check if any of the VMs you’ll create already exist and create only the ones that don’t exist.
Figure 7 will follow a simple algorithm to name the VMs using the vm_base_name field in the config.json and append the numbers 0 – (vm_count – 1) to the end of the VM name. Given the current config.json, create sedouardmachine0, sedouardmachine1 and sedouardmachine2 because none of those machines already exist on the subscription.
Figure 7 Create a List of Virtual Machines to Provision
scripty.invoke('vm list', function (err, result) {
if (err) {
return cb(err);
}
// Read the desired VM name from config.json.
var baseName = nconf.get('vm_base_name');
var vmNames = [];
// Create the array of the computed VM names.
for (var z = 0; z < count; z++) {
vmNames.push(baseName + z.toString());
}
// Go through the list of existing VMs.
for (var i in result) {
for (var k in vmNames) {
if (result[i].VMName === vmNames[k]) {
// A VM we intend on creating already exists on this sub.
// Remove it on the list of VMs to create.
delete vmNames[k];
}
}
}
// vmNames now only contains the name of VMs that do not exist.
// Create them.
Figure 7 confirms the list of non-existing VMs in the vmNames array. Figure 8 will use the async library to manage the asynchronous tasks that will go off and create the VMs.
Figure 8 Finish Provisioning Virtual Machines
var domainName = nconf.get('dns_name');
var userName = nconf.get('vm_username');
var password = nconf.get('vm_password');
var imageName = nconf.get('vm_image_name');
var vmCreationTasks = [];
var taskArguments = [];
for (var m in vmNames) {
if (vmNames[m]) {
var cmd = {
command: 'vm create',
positional: [vmNames[m], imageName, userName, password],
'affinity-group': nconf.get('affinity_group').name,
'virtual-network-name': nconf.get('vnet_name')
}
// Define the async task function that will create each VM.
var task = function (args, cb) {
console.log('Creating vm ' + vmNames[args[0]]);
scripty.invoke(args[1], function (err) {
if (err) {
console.log('Vm creation failed: ' + vmNames[args[0]]);
return cb(err);
}
console.log('vm creation of ' + vmNames[args[0]] + ' successful');
cb();
});
}
// Bind this function to this context and pass the index and VM create command.
task = task.bind(this, [m, cmd]);
vmCreationTasks.push(task);
}
}
// Execute each VM creation task using the async libray.
async.series(vmCreationTasks, function (err) {
if (err) {
return cb(err);
}
console.log('All VMs created successfully!');
cb();
})
Figure 8 creates a task for each VM you need to create. Then you need to add those tasks to an array, vmCreationTasks. Each task function is bound to the necessary arguments in order to call the Azure azure-cli with the correct VM name. Afterward, use the async library to run each task in series and execute the callback with a null or an error parameter indicating if the VMs were successfully created. Figure 9 confirms the output of the script and lists out all the VM that have been created.
.png "The Final List of Virtual Machines")
Figure 9 The Final List of Virtual Machines
Each of your VMs has been successfully created. You can even check back to your portal to confirm that the machines have been created and are part of the specified virtual network. Use the GitHub repository at bit.ly/azure-xplat-cli to get started scripting your Azure setup using Node.js.
Wrapping Up
The Azure xplat-cli lets you completely automate the process of configuring and deploying Azure resources in portable scripts you can use on virtually any OS. It also lets you use source control and configuration files to simplify sharing. You could have multiple versions of config.json, each one representing a deployment model for test, staging and production.
You can also recreate the deployment process quickly and efficiently. The forthcoming Azure Resource Manager Microsoft will release in 2015 will go above and beyond what’s illustrated here, but this approach is virtually identical to what the Azure Resource Manager will do—which is declaratively specify how resources are provisioned in the Azure datacenter.
Steven Edouard is a developer evangelist at Microsoft. Before that, he worked on the .NET runtime team as a software test engineer delivering products like the Microsoft .NET Framework 4.5 and .NET Native Compilation. Now his passion resides in exciting people on the limitless potentials of cloud computing services through engaging technical demonstrations, online content and presentations.
Thanks to the following Microsoft technical expert for reviewing this article: Bruno Terkaly