• 아티클

October 2014

Volume 29 Number 10

Azure Insider : Connect Your IoT Device to the Cloud

Bruno Terkaly | October 2014

Bruno Terkaly We recently introduced the idea of connecting an Internet of Things (IoT) device—in our case the Raspberry Pi—to the cloud (Microsoft Azure) to deliver a push notification to your mobile device whenever somebody rings your doorbell at home. This lets you see who’s at your front door step at home from anywhere in the world using your mobile device.

In the September 2014 article, “Soup to Nuts: From Raw Hardware to Cloud-Enabled Device” (msdn.microsoft.com/magazine/dn781356), we walked through the process of integrating the device to storage blobs using Azure Mobile Services to obtain a Shared Access Signature (SAS). This let the device directly upload the file to a storage container without any service tier. Now we can upload the photo to the cloud, but there are still a few things left to do to send a notification to the device.

So far, Azure Mobile Services and Azure Storage have been leveraged in this project. You’ll also need to take advantage of the service bus queues and scheduled tasks services to send messages from the Raspberry Pi device to your mobile device. These two services will be integrated with Azure Mobile Services, which will receive messages from the service bus queue and record them in a database. In the spirit of open source and new-age solutions, we’ll use a MongoDB database hosted and maintained by MongoLab that you can add for free as an Azure Add-on.

Service Bus for Distributed Messaging

In the context of IoT computing, the service bus queueing capabilities provide powerful abstractions. Service bus queueing supports asynchronous messaging between endpoints and can scale to support virtually any workload. It also lets applications communicate between the cloud and on-premises without requiring virtual networking, which can be a logistical and security nightmare.

We addressed some of these issues in the February 2014 column, “The Windows Azure Service Bus and the Internet of Things” (msdn.microsoft.com/magazine/dn574801). Azure offers another queueing service called storage queues, which has similar capabilities, but differs in a few key ways. We chose to go with service bus queues because of publisher/subscriber capabilities, greater scalability in message handling and the ability to easily convert to other service bus services. You can read more about storage queues at bit.ly/XSJB43.

Service bus queues expose a regular RESTful API, which supports the ability to receive messages by long polling. This is a technique of opening an HTTP connection for a certain period of time. Long polling is a great technique for IoT computing scenarios because it supports timeouts. This lets devices close connections until the next long poll, providing relief to power consumption and network resources.

We’ll have a many-to-one relationship of SmartDoor doorbell devices to a single mobile service. More specifically, this form of pub/sub will involve many message publishers and only a single message subscriber. The subscriber in this case will be the mobile service, which will read from the queue. One or more SmartDoor devices will be the publisher. It’s entirely possible to do the reverse, in case you’d like to send a message back to the device.

There are four types of communication patterns typically used in IoT scenarios. As you can see in Figure 1, one or more Raspberry Pi devices are publishing messages to the service bus queues. Azure Mobile Services is the only subscriber to service bus queues.

Architectural Diagram of Azure Mobile Services Reading from Service Bus
Figure 1 Architectural Diagram of Azure Mobile Services Reading from Service Bus

Azure Mobile Services provides an integrated task scheduler, which can schedule tasks at fixed intervals. We’ll read from the service bus queue at an interval of 60 seconds. Check our blog post at bit.ly/1o1nUrY for more detailed information on connecting mobile services to a service bus queue. With our scheduled task, doorBellListener (see Figure 2), we’ll read from the queue by using the service bus API in the Node.js Azure SDK with our connection string.

Figure 2 Azure Mobile Scheduled Task doorbellListener

// Get a reference to the azure module
var azure = require('azure');
// Get our service bus connection string
var connectionString = process.env.ServiceBusConnectionString;
// This task will run for 60 seconds so the initial timeout should be 60
var c_Timeout = 60;
function doorbellListener() {
  //Get the current unix time in seconds
  var date = new Date();
  var time = date.getTime();
  var startSeconds = time / 1000;
  var sb = azure.createServiceBusService(connectionString);
  listenForMessages(c_Timeout);
  // Define a function that will listen for messages
  // on the queue for number of seconds
  function listenForMessages(seconds) {
    console.log('Doorbell Listener Started for timeout: ' + seconds);
    // Long poll the service bus for seconds
    sb.receiveQueueMessage("smartdoor", { timeoutIntervalInS: seconds },
      function(err, data) {
        if(err){
          // This path will get hit if we didn't get any messages
          console.log(err);
        }
        else{
          // We have received a message from a device
          var ringNotification = JSON.parse(data.body);
          console.log('recieved message from doorbell ' +
            ringNotification.doorbellId + '
            with image link ' + ringNotification.imagePointer)
          function continueListeningForMessages(){
            // Go back and listen for more messages for the duration of this task
            var currentDate = new Date();
            var currentSeconds = currentDate.getTime() / 1000;
          console.log('currentSeconds ' + currentSeconds);
          console.log('startSeconds ' + startSeconds);
        // Compute the seconds between when we started this scheduled task and now
        // This is the time that we will long-poll the service bus
        var newTimeout = Math.round((c_Timeout - (currentSeconds - startSeconds)));
          if(newTimeout > 0){
          // Note: the receiveQueueMessage function takes ints no decimals!!
          listenForMessages(newTimeout);
        }
      }

Because of the asynchronous nature of Node.js and the behavior of sending a message to the service bus via long polling, you must compute the timeout value that’s passed to recieveQueueMessage, depending on how long this instance of the scheduled task has been running. This will prevent multiple instances of the task from running simultaneously.

The nice thing about service bus is it exposes a RESTful API. No matter what device you’re using, you can send it messages. Azure has SDKs for major languages such as Python, Ruby, Node.js and C#. Because we want the code to be translated to any platform, we’ll use the RESTful API directly.

In order to send a message, we’ll need to create a policy for the service bus queue. This policy is a key that permits certain operations on your service bus. There are a certain number of policies for any given queue. Take that into account in your own problem domain.

We’ll generate a policy called DevicePolicy that only lets users send messages to the service bus queue (see Figure 3).

This ensures that if the key ever gets into the wrong hands, nobody else can listen to messages on the service bus.

A DevicePolicy Will Send Messages to the Service Bus Queue
Figure 3 A DevicePolicy Will Send Messages to the Service Bus Queue

The device code for the Raspberry Pi to send a message the service bus queue is shown in Figure 4.

Figure 4 Sending a Message Indicating Successful Photo Upload

Console.WriteLine("Sending notification to service bus queue");
  WebRequest sbRequest = WebRequest.Create(
    "https://smartdoordemo.servicebus.Windows.net/smartdoorqueue/messages");
  var headers = sbRequest.Headers;
  sbRequest.Method = "POST";
  using (var sbMessageStream = sbRequest.GetRequestStream())
  {
    string body = JsonConvert.SerializeObject(new DoorBellNotification()
    {
      doorBellID = deviceID,
      imageUrl = photoResp.photoId
    });
    var bytes = Encoding.UTF8.GetBytes(body);
    sbMessageStream.Write(bytes, 0, bytes.Length);
    headers.Add("Authorization", createToken(
      "https://smartdoordemo.servicebus.Windows.net/smartdoorqueue/
      messages", "DevicePolicy",
      ConfigurationManager.AppSettings["ServiceBusSharedAccessKey"]));
  }
  try
  {
    Console.WriteLine("Sending door bell notification for " + deviceID);
    using (var response = sbRequest.GetResponse())
    {
      Console.WriteLine("Sucessfully Sent");
    }
  }
  catch (Exception e)
  {
    Console.WriteLine("Couldn't post to service bus -" + e);
  }

You can see the full code of program.cs at bit.ly/1qFYAF2. In that code, we do a POST request on the service bus queue RESTful API and provide an SAS, similar to the signature we received from mobile services for blob storage. This SAS is composed of a cryptographic key using the SHA-256 algorithm. It defines the resource being accessed as well as the SAS expiration time. The createToken method is a simple method that constructs the SAS based on your Shared Access Key from the DevicePolicy policy.

After constructing the SAS, we place it in an HTTP header and place the serialized message (in JSON format) in the request body. After the POST request is made, a message is sent to the service bus queue to indicate the successful upload of a photo. The message contains the link to the uploaded blob and a unique identifier for this doorbell device which comes from the application settings. This identifier can be found in the app.config xml file, which sits next to the doorbell listener executable:

<appSettings>
  <add key="DoorBellID" value="123456"/>
...
</appSettings>

Now the device is sending messages to the service bus when it runs. By going to the Log tab in our Mobile Services landing page we can access the live service log output. When we run the C# code in Visual Studio you can see the mobile service got the message contents through the log output in the portal.

Although Azure Mobile Services offers perfectly good tables (backed by SQL Server) for storage, we’re going for a slightly different approach. We’ll use MongoDB as our database solution. MongoDB works very well with Node.js because it’s a document-­oriented database and resembles a collection of JSON objects in storage. Read more about the MongoDB open source project at mongodb.org. We’ll use MongoLab, a Database as a Service (DaaS) provider, to host our database.

We need the database to keep track of a couple things:

  • DoorBells—An individual Raspberry Pi device
  • Pictures—An individual picture taken by a doorbell

Once the MongoDB database is set up, we can use Mongoose as our MongoDB Driver for our Azure Mobile Services Node.js code by installing it to our mobile services Git repository. This is the same process we went through to install the qs node module:

npm install mongoose

Pushing the local repository will trigger the mobile service to install Mongoose to its repository. This is how we can get any Node.js package into Azure Mobile Services. As with every Node.js module, we can reference it using RequireJs and initialize Mongoose with our MongoDB connection string in our scheduled task:

var mongoose = require('mongoose');

Mongoose will be responsible for creating a structured schema in our document database (see Figure 5). We’ll use two entities: doorbell and photo. Every doorbell object in our database will represent one Raspberry Pi device. It will contain a unique identifier, doorBellID, and an array of photo objects. Each photo contains a link to the photo in blob storage, as well as a timestamp generated by the service when it receives a service bus message.

Figure 5 Schemas Defined in Mongoose

var photoSchema = mongoose.Schema({
                url : String,
                timestamp: String
            });
var doorbellSchema = mongoose.Schema({
            doorBellID: String,
            photos: [photoSchema]
        });
var Photo = mongoose.model('Photo', photoSchema)
var DoorBell = mongoose.model('DoorBell', doorbellSchema);
// Expose these schemas
exports.DoorBell = DoorBell;
exports.Photo = Photo;

We expose the DoorBell and Photo schemas publicly via the exports keyword. Notice how photoSchema is interleaved within doorbellSchema. This is how the data will be reflected when stored in the database.

We’ll place this code in the shared folder of our mobile services repository. This lets us use the schemas anywhere in our service. To reference the schemas in our scheduled task, we just need to import it with a require statement:

var schemas = require('../shared/schemas.js');

Now we can use these schemas in our scheduled task to define new database entities. We’ll use a specialized function to ensure we’re connected to MongoDB and execute a callback. Upon receiving a message from the service bus, the mobile service should:

  1. Check if the doorbell with the specified doorBellID in the message exists in the database.
  2. If it doesn’t exist, create a new doorbell entity with the photo.
  3. If it does exist, just append the new photo to the existing doorbell’s photo array.

You can see the logic for this in the code in Figure 6.

Figure 6 Logic to Verify Doorbell and Manage Photo

// Create database entry for this image
dbConnectAndExecute(function(err){
  if(err){
    console.log('could not record image to database -' + err);
    return;
  }
  DoorBell.findOne({ doorBellID: ringNotification.doorBellID},
    function(err, doorbell){
    if(err){
      return console.error(err);
    }
    if(doorbell === null){
      console.log('Doorbell not found in DB, creating a new one');
      // Take the entire body's json, assuming it fits into this schema
      var entityObject = {
        doorBellID : ringNotification.doorBellID,
        photos : []
      };
      entityObject.photos.push({
        url : ringNotification.imageUrl,
        // Set timestamp to current server time
        timestamp : ((new Date()).getTime()).toString()
      })
      var doorbell = new DoorBell(entityObject);
    }
    else{
      // We already have this device in the database—add a picture
      doorbell.photos.push({
        url : ringNotification.imageUrl,
        // Set timestamp to current server time
        timestamp : ((new Date()).getTime()).toString()
      });
    }
    // Commit changes to db
    doorbell.save(function (err, entity) {
    if(err){
      return console.error(err);
    }
    console.log('sucessfully created new entity for: ' + entity);
    return;
  });
  });
});

You can see a full demo of doorbelllistener at bit.ly/VKrlYU.

We call the dbConnectAndExecute function to ensure we’re connected to our MongoDB database in MongoLabs. Then we query the database for a doorbell with the ID specified in the message. If the query comes up empty, we create a new doorbell entity. Otherwise, we append the photo to the fetched entity and commit the changes to the database.

Now see in Figure 7 what happens when we send a photo and service bus message with our Raspberry Pi device.

The Outcome of Sending a Photo and Message
Figure 7 The Outcome of Sending a Photo and Message

Inspecting the mobile services log in the portal after Raspberry Pi has sent the message shows the photo was processed and added to the database.

For good measure, we can check our MongoDB database to ensure we’re actually tracking the photos. MongoLab provides a rich MongoDB inspection interface via its Web portal. Figure 8 shows what the doorbell entity may look like after a couple photo uploads.

MongoDB Registering Entries
Figure 8 MongoDB Registering Entries

Now, Raspberry Pi is fully integrated to our cloud service. The device can upload a file directly to cloud storage, notify the cloud service via service bus and have the information stored in a database.

The next step in this series will be integrating a mobile application to the cloud back end with push notifications of the images. We’ll use custom APIs to surface objects from the database to the app and integrate a third-party API for image facial recognition and a notification hub to power the push notifications. You can examine the code repositories to build and tweak the solution yourself. The Azure Mobile Services back end is at bit.ly/1mHCl0q. You’ll find the Raspberry Pi Code at bit.ly/1l8aOtF.

Integrating Raspberry PI devices to cloud back ends is critical. Leveraging open source technologies such as Node.js makes sense in many scenarios where you’re using a lightweight back end to scale to a large number of devices.

The Azure service bus provides a safe and convenient way for occasionally connected devices to leverage a reliable messaging infrastructure that can scale to many clients. Finally, leveraging NoSQL stores is a popular way to persist data and retain native JavaScript syntax in both the data layer and back-end Node.js service.

Steven Edouard is a developer evangelist at Microsoft. Before that, he worked on the .NET runtime team as a software test engineer delivering products like the .NET Framework 4.5 and .NET Native Compilation. Now his passion resides in exciting people on the limitless potentials of cloud computing services through technical demonstrations, online content and presentations.

Bruno Terkaly is a developer evangelist for Microsoft. His depth of knowledge comes from years of experience in the field, writing code using a multitude of platforms, languages, frameworks, SDKs, libraries and APIs. He spends time writing code, blogging and giving live presentations on building cloud-based applications, specifically using the Microsoft Azure platform. You can read his blog at blogs.msdn.com/b/brunoterkaly.

Thanks to the following Microsoft technical experts for reviewing this article: Gil Isaacs and Brent Stineman