Create an End-to-end IOT Scenario with Azure Services

  • Article

IOT or Internet of things is taking over the world and we are encountering more and more of it in our daily lives. But when we come to think of it what does the word IOT actually mean? What are these IOT solutions and how do we start building them?

To answer these questions, lets first start with defining what is IOT? There is no standard definition for IOT but moreover it is any solution that essentially has following four components.

  1. Things - Things can be anything that can send some kind of useful data. It can be a simple phone or a complex array of sensors sending the data.
  2. Connectivity - These things need a way to communicate with each other and with internet and for that they need connectivity and protocols.
  3. Data- The data can be a simple telemetry data, an image or even a video feed.
  4. Analytics- The data that is gathered needs to be analyzed to derive insights from it. These analytics can be real time analytics such as sending an alert or taking an action or a long term analytics where you might use machine learning on historical data to detect a pattern or predict an outcome.

So how would a typical event processing would happen in an IOT Scenario. Lets a take a look.

image1

Starting with producers, these are your devices or things which would be sending data. The data can either be directly sent to cloud if the devices are internet enabled. If the devices are legacy devices and not internet enabled, then some kind of gateways or aggregators can be used to collect the data from all such devices and then send it to cloud. Then you need an ingestion mechanism - a service or solution which is capable of ingesting data from any number of devices at a time and scale as per the need. Next, you would need a service which would process the incoming data, either to generate a real time action or to aggregate it and store it for further analysis. You would also need  data storage capabilities to store the huge amount of data that will be coming in and at the last you might want to have a way to visualize, search or query this data or to use it for your machine learning experiments.

How Azure fits into the picture ?

Microsoft Azure services give you a way to create your entire IOT solution with the use of many different components. Its not a one way solutions but rather a platform consisting of different services from which you can choose the services best fitted for your own scenario. Here is just a glimpse of the all the available services that can be used in your IOT Scenarios.

Microsoft Services

Let's build a scenario, shall we?

We will create a simple IOT scenario, where we will gather data from different devices and ingest it in cloud. Once it is ingested, we will analyze the data to detect an anomaly and then raise an alert. We will use following Azure services in this scenario.

Scenario

Step 1 - Devices

Azure IOT Hub has a way to interface with many different types of devices. This repository contains both IoT device SDKs and IoT service SDKs. Device SDKs enable you connect client devices to Azure IoT Hub. Service SDKs enable you to manage your IoT Hub service instance. https://github.com/Azure/azure-iot-sdks

The SDKs are available for different programming languages such as C, Python, Node.js, Java, .Net and also can be used with a broad range of OS platforms and devices such as Linux, Windows and real-time operating systems. The other way to connect to IOT Hub is via IOT Hub Rest APIs if for some reason you can not use a SDK.

In this scenario, we are going to simulate a device using SDK for .NET.

This simulator will generate random values in a given range and send those values to Azure IOT Hub at a specific Interval.

The Link to Code https://github.com/gsamant/IOTEndtoEndDemo

Step 2- Create and setup Azure IOT Hub

Azure IOT Hub helps you to establish bi-directional communication with billions of IOT devices and to set up individual identities and credentials for each of your connected devices.

(You need an active Azure subscription to create Azure IOT Hub)

Create an IOT hub using steps described here. (For this scenario, follow the steps only till the creation of IOT Hub.)

Once your IOT Hub is created, the next step is to register a device, this can be done via Azure SDKs, Rest APIs or Azure CLI. You can also use the tools such as Device Explorer (Windows) or IOT Hub  Explorer.

Once the device is registered you will have a device connection string which will have IOT Hub Name, Device ID and a Shared Access Key unique to your device. Once you have the connection string, use the values from the connection string to update the following lines in Program.cs file

static string iotHubUri = "<IOT Hub URI>";
static string deviceId = "<Device ID>";
static string deviceKey = "<Device Shared access key>";

Step 3- Create and setup Azure Event Hub

Azure Event hub isa managed service that can intake and process massive data streams from websites, apps and devices. This service can be used to directly used to ingest data from devices as well however unlike IOT hub, Event Hub can handle only device to cloud communications.

In this scenario however, we are going to use Azure event hub as an output for the Azure stream analytics job.

Create an Event hub using steps described here. And Note the connection string.

Step 4 - Create a Stream Analytics job

Azure Stream Analytics is  a fully managed real-time event-processing engine. Events can come from sensors, applications, devices, operational systems, websites, and a variety of other sources.

Create a Stream Analytics job using steps described here. (For this scenario, follow the steps only till the creation of Stream analytics job). You will have an empty job as below

Stream Analytics

Once the Stream analytics job is created, Add the input, output and update the Query as shown below.

[caption id="attachment_135" align="alignleft" width="300"] Input[/caption]

[caption id="attachment_145" align="alignleft" width="300"]Output Output[/caption]

[caption id="attachment_155" align="alignnone" width="1024"]Query Query[/caption]

Start the Stream Analytics Job.

Step 5 - Create an Azure function to process the Event hub queue

Azure Function App is a serverless event based Compute service. The processing can be triggered via an event, a timer or a SaaS activity. In this scenario, we will create an Azure function which will be triggered when the event hub defined in Step 4 receives an event.

  1. Go to the Azure portal and sign-in with your Azure account.

  2. Search for Function App and Click Create.

    [caption id="attachment_165" align="alignnone" width="300"]Azure Function Creation Azure Function Creation[/caption]

  3. In your function app, click + New Function > EventHubTrigger - C# > Create. This creates a function with a default name that is run on the trigger from an Event Hub. It will also ask you to add the name of your Event Hub and the Event hub connection string.(For connection string click new, give a name to your connection and paste the connection string from Step 4)

  4. Copy the code from ProcessEvent.txt file from the GitHub link

    [caption id="attachment_175" align="alignnone" width="800"]Azure Function Code Azure Function Code[/caption]

  5. Leave the Client.BaseAddress URL blank for now. We will update this once we create the Logic App.

Step 6 - Create an Azure Logic app to send an Alert

Azure Logic Apps enable you to develop integration solutions with ease and let you automate and simplify business workflows across on-premises and the cloud.

In this Scenario we will create a Logic App which will be triggered from an HTTP request and will send SMS using a Twilio Connector.

  1. On the Azure portal dashboard, select New.

  2. In the search bar, search for 'logic app', and then select Logic App. You can also select New, Web + Mobile, and select Logic App.

  3. Enter a name for your logic app, select a location, resource group, and select Create. If you select Pin to Dashboard the logic app will automatically open once deployed.

  4. After opening your logic app for the first time you can select from a template to start. For now click Blank Logic App to build this from scratch.

  5. The first item you need to create is the trigger. This is the event that will start your logic app. Search for Request in the trigger search box, and select it.

  6. In the Request Body JSON Schema field, paste following Schema.

    [caption id="attachment_185" align="alignnone" width="600"]JSON Schema JSON Schema[/caption]

  7. After you Click Save. The HTTP Post url will be generated. Copy this URL and paste it in Client.BaseAddress URL in Azure Function that is created previously.

    [caption id="attachment_195" align="alignnone" width="700"]Logic App -Request Logic App -Request[/caption]

     

  8. Before going for the next step, we will need to create a Twilio Account. You can create a Free trial Account here. Before moving to next step, you need to have a verified Twilio Phone number that can send/receive SMS.

  9. Click on New Step and search for Twilio. Select Twilio-Send Test Message (SMS)

  10. Enter a connection Name for Twilio, Twilio Account ID and Twilio Access Token (You can get these two from your Twilio dashboard.)

    [caption id="attachment_205" align="alignnone" width="700"]Twilio Account Setup Twilio Account Setup[/caption]

  11. Once the connection is saved. Enter the to and from Phone numbers and Text for message.

    [caption id="attachment_215" align="alignnone" width="700"]Twilio SMS Setup Twilio SMS Setup[/caption]

  12. Save the Logic App changes.

So far, we are done with the setup of our scenario. Lets test the scenario.

Testing the scenario

  1. Run the SendEvent application. You will see events going to the IOT hub.

    [caption id="attachment_225" align="alignnone" width="600"]App Run App Run[/caption]

    Stop the application and in Program.cs file, uncomment the following lines. This will allow you to manually enter the Temperature value more that 60.

    Code snippet

    [caption id="attachment_235" align="alignnone" width="600"]App Run App Run[/caption]

  2. Check the SMS Alert on your phone.

    [caption id="attachment_265" align="alignnone" width="200"]SMS Alert SMS Alert[/caption]

  3. Also, check the Azure Function log.

    [caption id="attachment_245" align="alignnone" width="800"]Azure Function Log Azure Function Log[/caption]

     

Recap

We created a simple scenario where we simulated a device sending telemetry information. Then we used IOT Hub to ingest the data from the device and Stream analytics job to process the data in real time. The stream analytics query, checked if the temperature is going above a threshold and when it did it sent that event to an Event hub.

We then created an Azure function which will get triggered as soon as there is a new event on the Event hub and will send the Event data to a Logic App using HTTP Request Endpoint. On the Logic App trigger, we created a Twilio connector which will read the temperature from the payload and send the Alert.