March 2014

Volume 29 Number 3

Windows Azure Insider : The Windows Azure Service Bus and the Internet of Things, Part 2

Bruno Terkaly

Bruno Terkaly and Ricardo VillalobosIn our last column (msdn.microsoft.com/magazine/dn574801), we discussed the current technology landscape for machine-to-machine (M2M) computing, which refers to technologies that interconnect devices, usually for industrial instrumentation, in the form of sensors or meters. The proliferation of affordable and easy-to-program tiny computers has expanded this concept into what’s called the Internet-of-Things (IoT), opening the door to scenarios where even ordinary home appliances can be controlled or used as sources of information to generate events. This way, it isn’t difficult to send alerts when it’s time to replenish the fridge, automatically close the window blinds as night falls or set the thermostat based on the family habits.

We also made the case for using the Windows Azure Service Bus for device connectivity, as an alternative to using a VPN, when trying to solve the addressability, security, and performance concerns associated with deploying a large number of sensors or meters. This is becoming increasingly relevant considering that, according to the latest BI Intelligence report from Business Insider, there will be more than 9 billion connections directly related to the IoT by the year 2018 (read.bi/18L5cg8).

Using a designated Service Bus queue or topic for a device provides an elegant way to incorporate resiliency and occasional connectivity for IoT applications. In this article, we’ll walk through a hands-on Windows Azure implementation that illustrates these concepts, designing a Service Bus blueprint with device queues, deploying a listening worker role in Cloud Services, and programming an Arduino device that executes commands sent remotely by mobile clients, as shown in Figure 1.

An Internet-of-Things Architecture Using the Windows Azure Service Bus
Figure 1 An Internet-of-Things Architecture Using the Windows Azure Service Bus

If you look at the diagram, the Windows Azure Service Bus component becomes the centerpiece of the design, providing the authentication, message distribution and scalability to support the multiple devices that will be sending data or receiving remote commands. The Service Bus is available in all Microsoft datacenters that offer Windows Azure services, and it’s backed up by a highly redundant storage infrastructure. Also, like all other Windows Azure components, it offers an open and easy-to-understand REST interface, along with multiple SDKs (Microsoft .NET Framework, Java, PHP, Ruby, among others) built on top of it.

In our proposed architecture, devices “talk” to a .NET application running on Windows Azure Cloud Services, which acts as a gateway to the Service Bus in order to simplify the communication process with its assigned queue. This approach fully enables any of the four IoT communication patterns described in our previous column: Telemetry, Inquiry, Command and Notification. Here, we’ll implement a scenario in which a mobile device sends a command to another device in order to execute an action—in this case, turn an LED on or off. One of the benefits of this solution is that if the device is temporarily offline, it can pick up the commands whenever it reconnects to the Internet. You can also set up an expiration time in a message to avoid the execution of a task at an inconvenient moment or schedule messages to be sent at a specific time in the future.

For this example, we’ll use the well-known, well-documented Arduino device, as described in our previous column. For the mobile client portion of the proof-of-concept, we’ll create a Windows Phone application.

Here’s our simple scenario:

  1. When the Arduino device is started, it sends an identification signal to the gateway application running on Windows Azure Cloud Services. The gateway creates a Service Bus queue for the device in case it doesn’t exist, and establishes a TCP connection, ready to send commands.
  2. A Windows Phone application sends a command to the Windows Azure Service Bus queue assigned to the device.
  3. The message remains in the queue until the gateway application picks it up and sends the command to the Arduino device via the established TCP connection.
  4. The Arduino device turns the LED on or off based on the command.

Let’s look at the steps to make this happen, one by one.

Step 1: Create the Windows Azure Service Bus Namespace Using your Windows Azure credentials (you can request a trial account at bit.ly/1atsgSa), log in to the Web portal and click on the SERVICE BUS section (see Figure 2). Select the CREATE option, and enter a name for your namespace. Then, click on CONNECTION INFORMATION and copy the text in the Connection String box, which you’ll need later.

Creating the Windows Azure Service Bus Namespace
Figure 2 Creating the Windows Azure Service Bus Namespace

Step 2: Create the Gateway Application and Deploy to Windows Azure Cloud Services Code for the gateway application, which retrieves messages from the Service Bus queue and relays the commands to the Arduino device, is included with the code download (available at msdn.microsoft.com/magazine/msdnmag0314). It’s based on the work of Clemens Vaster, who kindly contributed his guidance and expertise to this article. His original project can be found at bit.ly/L0uK0v.

Before we dive into this code, be sure you have Visual Studio 2013 installed, along with version 2.2 of the Windows Azure SDK for .NET (bit.ly/JYXx5n). The solution includes three different projects:

  • ArduinoListener—contains the main WorkerRole code.
  • ConsoleListener—the console version of the ArduinoListener, for local testing.
  • MSDNArduinoListener—the Windows Azure deployment project for ArduinoListener.

If you inspect the ServiceConfiguration.cscfg files (for both cloud and local deployment) for the MSDNArduinoListener project, you’ll see a setting that stores the connection string for the Service Bus. Replace its value with the one obtained in Step 1. The rest is already configured for the solution to work, including the definition of port 10100 for receiving connections from the devices. Next, open the WorkerRole.cs file in the ArduinoListener project, where the main code is located.

There are four main sections to analyze.

First, a TcpListener is created, and connections from devices are accepted:

var deviceServer = new TcpListener(deviceEP);
deviceServer.Start(10);
try
{
  do
  {
    TcpClient connection = 
      await deviceServer.AcceptTcpClientAsync();
    if (connection != null)
    {      ...

Once a connection with the device has been established, a NetworkStream is defined and set to listening mode. The readBuffer variable will contain the identifier value sent by each Arduino device:

NetworkStream deviceConnectionStream = connection.GetStream();
var readBuffer = new byte[64];
if (await deviceConnectionStream.ReadAsync(readBuffer, 0, 4) == 4)
{
  int deviceId = 
    IPAddress.NetworkToHostOrder(BitConverter.ToInt32(readBuffer, 0));
  ...

Next, a queue is created based on the deviceId value (in case it doesn’t exist), and a message receiver object is defined (see Figure 3). Then, the device queue receiver is set to asynchronous mode to pull messages (commands from the queue). This queue will store commands sent by mobile devices, such as a Windows Phone.

Figure 3 Creating a Queue

var namespaceManager = 
  NamespaceManager.CreateFromConnectionString(
    RoleEnvironment.GetConfigurationSettingValue("serviceBusConnectionString"));
if (!namespaceManager.QueueExists(string.Format("dev{0:X8}", deviceId)))
{
  namespaceManager.CreateQueue(string.Format("dev{0:X8}", deviceId));
}
var deviceQueueReceiver = messagingFactory.CreateMessageReceiver(
  string.Format("dev{0:X8}", deviceId), ReceiveMode.PeekLock);
do
{
  BrokeredMessage message = null;
  message = await deviceQueueReceiver.ReceiveAsync();
  ...

When a message is received in the queue, its content is inspected and if it matches the “ON” or “OFF” commands, the information is written to the connection stream established with the device (see Figure 4).

Figure 4 Writing to the Connection Stream

if (message != null)
{
  Stream stream = message.GetBody<Stream>();
  StreamReader reader = new StreamReader(stream);
  string command = reader.ReadToEnd();
  if (command != null)
  {
    switch (command.ToUpperInvariant())
    {
      case "ON":
        await deviceConnectionStream.WriteAsync(OnFrame, 0, 
            OnFrame.Length);
        await message.CompleteAsync();
        break;
      case "OFF":
        await deviceConnectionStream.WriteAsync(OffFrame, 0, 
            OffFrame.Length);
        await message.CompleteAsync();
        break;
    }
  }
}

Notice that the message isn’t removed from the queue (message.Complete­Async) unless the writing operation to the device connection stream is successful. Also, in order to keep the connection alive, the device is expected to send a ping heartbeat. For this proof of concept, we aren’t expecting confirmation from the device when it receives the message. In a production system, however, this would be required to comply with the “command” pattern.

Step 3: Deploy the Arduino­Listener Windows Azure Project to Cloud Services Deploying the ArduinoListener to Windows Azure is extremely simple. In Visual Studio 2013, right-click on the MSDN­ArduinoListener project and select the Publish option. You’ll find specific instructions for the Publish Windows Azure Application Wizard at bit.ly/1iP9g2p. After completing the wizard, you end up with a cloud service located at xyz.cloudapp.net. Record this name, as you’ll need it when you create the Arduino client in the next step.

Step 4: Program the Arduino Device to Talk to the Gateway (Listener) Arduino devices offer a rich interface for performing network operations using a simple Web client object. For our prototype, we decided to use the Arduino Uno R3 model (bit.ly/18ZlcM8), along with its corresponding Ethernet shield (bit.ly/1do6eRD). To install, interact and program Arduino devices using Windows, follow the guide at bit.ly/1dNBi9R. You’ll end up with an easy-to-use IDE (called the Arduino application), where you can write programs (called sketches) using JavaScript, as shown in Figure 5.

The Arduino Application
Figure 5 The Arduino Application

Figure 6 shows the sketch for interacting with the Arduino Listener created in Step 3, and now deployed in Windows Azure.

Figure 6 The Arduino Device Code

#include <SPI.h>#include <Ethernet.h>#include <StopWatch.h>
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network,
// and it's optional if DHCP is enabled.
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xBC, 0xAE };
static const byte deviceId[] = { 0x00, 0x00, 0x00, 0x01 };
static const uint8_t ACK = 0x01;
static const int LED_PIN = 8;
int connected = 0;
EthernetClient client;
StopWatch stopWatch;
long pingInterval = 200000;
void setup() {  Serial.begin(9600);
  Serial.println("Initialized");
  Ethernet.begin(mac);
  pinMode(LED_PIN, OUTPUT);}
void turnLedOn(){  digitalWrite(LED_PIN, HIGH);}
void turnLedOff(){  digitalWrite(LED_PIN, LOW);}
void loop() {
        if ( connected == 0)  {
    Serial.println("Trying to connect");
    char* host = "xyz.cloudapp.net";
    client.setTimeout(10000);
    connected = client.connect(host, 10100);
    if (connected)     {
      Serial.println(
        "Connected to port, writing deviceId and waiting for commands...");
      client.write(deviceId, sizeof(deviceId));
      stopWatch.start();
    }
    else
    {
      Serial.println("Connection unsuccessful");
      client.stop();
      stopWatch.reset();
    }
  }
  if (connected == 1)
  {
    if (stopWatch.elapsed() > pingInterval)
    {
      Serial.println("Pinging Server to keep connection alive...");
      client.write(deviceId, sizeof(deviceId));
      stopWatch.reset();
       stopWatch.start();
    }
    byte buf[16];
    int readResult = client.read(buf, 1);
    if (readResult == 0)
     {
      Serial.println("Can't find listener, disconnecting...");
      connected = 0;
      stopWatch.reset();
    }
    else if (readResult == 1)
    {
      Serial.println("Data acquired, processing...");
      switch ( buf[0] )
      {
        case 1:
          Serial.println("Command to turn led on received...");
          turnLedOn();
          break;
        case 2:
           Serial.println("Command to turn led off received...");
          turnLedOff();
          break;
      }
      stopWatch.reset();
      stopWatch.start();
    }
  }
}

Sketches for the Arduino have two main sections: setup and loop. Instructions in the setup section are executed once, and this is where variables are initialized and connections established. In our example, the Ethernet client and related values are defined, a serial connection (for debugging purposes) is established, and the pin where the LED is connected is initialized as an output port.

Code in the loop section is executed constantly, and it includes two main blocks based on the status of the TCP connection between the Arduino device and the listener running in Windows Azure Cloud Services: connected or disconnected. When the connection is established for the first time, a stopWatch object is started to keep track of the time elapsed for the connection. Also, the device identifier is sent to the listener, to be used as the name of the queue where messages and commands will be stored.

The code block that handles the Arduino behavior after the connection has been established keeps track of the time elapsed since the connection was created, pinging the listener every 200,000 ms, to keep the connection alive when no commands are received. This code also tries to read data from the listener, putting the data into the buf array when it arrives. If a value of “1” is detected, the LED is turned on, if the value is “2,” the LED is turned off. The stopwatch object is reset after each command.

Once the sketch has been uploaded to the device, the code runs on the Arduino controller in an infinite loop, trying to connect to a cloud service. When connected, it forwards the device id so the cloud service knows to which device it’s talking. Then the code begins to read input from the cloud service, telling the device whether to turn on or off the LED light (in this case, it’s connected to digital port 8 of the device).

Step 5: Creating a Windows Phone Client to Send to Device Queue Interacting with the device is as simple as sending messages to the device queue. As we mentioned at the beginning of the article, the Windows Azure Service Bus provides a REST interface that lets you interact with it from multiple programming languages. Because there’s no official SDK for Windows Phone developers, we used one of the examples from the Windows Phone community, which shows how to authenticate and interact with the Service Bus using HTTP requests and the WebClient object. The source code is also included with the code download, in the Visual Studio 2013 project called MSDNArduinoClient. Figure 7 shows the client’s main screen, from which you send commands to the Arduino device.

The Windows Phone Client Interface
Figure 7 The Windows Phone Client Interface

Creating similar clients for other mobile devices (including iOS and Android) wouldn’t be difficult, as most of them provide libraries to generate REST commands using HTTP request clients. Moreover, it’s possible to directly interact with the Windows Azure Service Bus using traditional languages such as Java, PHP or Ruby, which simplifies this process. These SDKs are published under an open source license, and can be found at github.com/WindowsAzure.

Wrapping Up

Building an Internet-­of-Things architecture using the Windows Azure Service Bus to manage devices and services connections provides an easy way to secure, scale and address clients individ­ually without incurring costly VPN solutions, with the benefit of efficiently handling occasionally disconnected scenarios. Queues act as dedicated mailboxes where messages between devices and services are exchanged, supporting the different communication use cases and patterns commonly found in the field. Windows Azure provides a reliable, geo-distributed and robust infrastructure for deploying the services required with a high volume of interconnected sensors and meters—a trend that will continue to grow in the years ahead.


Bruno Terkaly is a developer evangelist for Microsoft. His depth of knowledge comes from years of experience in the field, writing code using a multitude of platforms, languages, frameworks, SDKs, libraries and APIs. He spends time writing code, blogging and giving live presentations on building cloud-based applications, specifically using the Windows Azure platform. You can read his blog at blogs.msdn.com/b/brunoterkaly.

Ricardo Villalobos is a seasoned software architect with more than 15 years of experience designing and creating applications for companies in multiple industries. Holding different technical certifications, as well as a master’s degree in business administration from the University of Dallas, he works as a cloud architect in the DPE Globally Engaged Partners team for Microsoft, helping companies worldwide to implement solutions in Windows Azure. You can read his blog at blog.ricardovillalobos.com.

Terkaly and Villalobos jointly present at large industry conferences. They encourage readers of Windows Azure Insider to contact them for availability. Terkaly can be reached at bterkaly@microsoft.com and Villalobos can be reached at Ricardo.Villalobos@microsoft.com.

Thanks to the following Microsoft technical experts for reviewing this article: Abhishek Lal and Clemens Vasters