Explore Azure IoT Edge architecture on Linux

This article provides a detailed walkthrough of the Hello World sample code to illustrate the fundamental components of the Azure IoT Edge architecture. The sample uses the Azure IoT Edge to build a simple gateway that logs a "hello world" message to a file every five seconds.

This walkthrough covers:

  • Concepts: A conceptual overview of the components that compose any gateway you create with the IoT Edge.
  • Hello World sample architecture: Describes how the concepts apply to the Hello World sample and how the components fit together.
  • How to build the sample: The steps required to build the sample.
  • How to run the sample: The steps required to run the sample.
  • Typical output: An example of the output to expect when you run the sample.
  • Code snippets: A collection of code snippets to show how the Hello World sample implements key IoT Edge gateway components.

Azure IoT Edge concepts

Before you examine the sample code or create your own field gateway using IoT Edge, you should understand the key concepts that underpin the architecture of IoT Edge.

IoT Edge modules

You build a gateway with the Azure IoT Edge by creating and assembling IoT Edge modules. Modules use messages to exchange data with each other. A module receives a message, performs some action on it, optionally transforms it into a new message, and then publishes it for other modules to process. Some modules might only produce new messages and never process incoming messages. A chain of modules creates a data processing pipeline with each module performing a transformation on the data at one point in that pipeline.

A chain of modules in gateway built with Azure IoT Edge

IoT Edge contains the following:

  • Pre-written modules which perform common gateway functions.
  • The interfaces a developer can use to write custom modules.
  • The infrastructure necessary to deploy and run a set of modules.

The SDK provides an abstraction layer that enables you to build gateways to run on a variety of operating systems and platforms.

Azure IoT Edge abstraction layer

Messages

Although thinking about modules passing messages to each other is a convenient way to conceptualize how a gateway functions, it does not accurately reflect what happens. IoT Edge modules use a broker to communicate with each other, they publish messages to the broker (bus, pubsub, or any other messaging pattern) and then let the broker route the message to the modules connected to it.

A module uses the Broker_Publish function to publish a message to the broker. The broker delivers messages to a module by invoking a callback function. A message consists of a set of key/value properties and content passed as a block of memory.

The role of the Broker in Azure IoT Edge

Message routing and filtering

There are two ways of directing messages to the correct IoT Edge modules. A set of links can be passed to the broker so the broker knows the source and sink for each module, or the module can filter on the properties of the message. A module should only act upon a message if the message is intended for it. The links and message filtering is what effectively creates a message pipeline.

Hello World sample architecture

The Hello World sample illustrates the concepts described in the previous section. The Hello World sample implements a IoT Edge gateway that has a pipeline made up of two IoT Edge modules:

  • The hello world module creates a message every five seconds and passes it to the logger module.
  • The logger module writes the messages it receives to a file.

Architecture of Hello World sample built with Azure IoT Edge

As described in the previous section, the Hello World module does not pass messages directly to the logger module every five seconds. Instead, it publishes a message to the broker every five seconds.

The logger module receives the message from the broker and acts upon it, writing the contents of the message to a file.

The logger module only consumes messages from the broker, it never publishes new messages to the broker.

How the broker routes messages between modules in Azure IoT Edge

The figure above shows the architecture of the Hello World sample and the relative paths to the source files that implement different portions of the sample in the repository. Explore the code on your own, or use the code snippets below as a guide.

How to build the sample

Before you get started, you must set up your development environment for working with the SDK on Linux.

  1. Open a shell.
  2. Navigate to the root folder in your local copy of the iot-edge repository.
  3. Run the tools/build.sh script. This script uses the cmake utility to create a folder called build in the root folder of your local copy of the iot-edge repository and generate a makefile. The script then builds the solution, skipping unit tests and end to end tests. Add the --run-unittests parameter if you want to build and run the unit tests. Add the --run-e2e-tests if you want to build and run the end to end tests.
Note

Every time you run the build.sh script, it deletes and then recreates the build folder in the root folder of your local copy of the iot-edge repository.

How to run the sample

  1. The build.sh script generates its output in the build folder in your local copy of the iot-edge repository. This output includes the two IoT Edge modules used in this sample.

    The build script places liblogger.so in the build/modules/logger/ folder and libhello_world.so in the build/modules/hello_world/ folder. Use these paths for the module path value as shown in the following example JSON settings file.

  2. The hello_world_sample process takes the path to a JSON configuration file a command-line argument. The following example JSON file is provided in the SDK repository at samples/hello_world/src/hello_world_lin.json. This configuration file works as is unless you modify the build script to place IoT Edge modules or sample executables in non-default locations.

    Note

    The module paths are relative to the current working directory from where the hello_world_sample executable is launched, not the directory where the executable is located. The sample JSON configuration file defaults to writing 'log.txt' in your current working directory.

    {
        "modules" :
        [
            {
              "name" : "logger",
              "loader": {
                "name": "native",
                "entrypoint": {
                  "module.path": "./modules/logger/liblogger.so"
                }
              },
              "args" : {"filename":"log.txt"}
            },
            {
                "name" : "hello_world",
              "loader": {
                "name": "native",
                "entrypoint": {
                  "module.path": "./modules/hello_world/libhello_world.so"
                }
              },
                "args" : null
            }
        ],
        "links":
        [
            {
                "source": "hello_world",
                "sink": "logger"
            }
        ]
    }
    
  3. Navigate to build folder.
  4. Run the following command:

    ./samples/hello_world/hello_world_sample ./../samples/hello_world/src/hello_world_lin.json

Typical output

The following is an example of the output written to the log file by the Hello World sample. The output is formatted for legibility:

[{
    "time": "Mon Apr 11 13:48:07 2016",
    "content": "Log started"
}, {
    "time": "Mon Apr 11 13:48:48 2016",
    "properties": {
        "helloWorld": "from Azure IoT Gateway SDK simple sample!"
    },
    "content": "aGVsbG8gd29ybGQ="
}, {
    "time": "Mon Apr 11 13:48:55 2016",
    "properties": {
        "helloWorld": "from Azure IoT Gateway SDK simple sample!"
    },
    "content": "aGVsbG8gd29ybGQ="
}, {
    "time": "Mon Apr 11 13:49:01 2016",
    "properties": {
        "helloWorld": "from Azure IoT Gateway SDK simple sample!"
    },
    "content": "aGVsbG8gd29ybGQ="
}, {
    "time": "Mon Apr 11 13:49:04 2016",
    "content": "Log stopped"
}]

Code snippets

This section discusses some key sections of the code in the hello_world sample.

IoT Edge gateway creation

The developer must write the gateway process. This program creates the internal infrastructure (the broker), loads the IoT Edge modules, and sets everything up to function correctly. IoT Edge provides the Gateway_Create_From_JSON function to enable you to bootstrap a gateway from a JSON file. To use the Gateway_Create_From_JSON function, you must pass it the path to a JSON file that specifies the IoT Edge modules to load.

You can find the code for the gateway process in the Hello World sample in the main.c file. For legibility, the following snippet shows an abbreviated version of the gateway process code. This example program creates a gateway and then waits for the user to press the ENTER key before it tears down the gateway.

int main(int argc, char** argv)
{
    GATEWAY_HANDLE gateway;
    if ((gateway = Gateway_Create_From_JSON(argv[1])) == NULL)
    {
        printf("failed to create the gateway from JSON\n");
    }
    else
    {
        printf("gateway successfully created from JSON\n");
        printf("gateway shall run until ENTER is pressed\n");
        (void)getchar();
        Gateway_LL_Destroy(gateway);
    }
    return 0;
}

The JSON settings file contains a list of IoT Edge modules to load and the links between the modules. Each IoT Edge module must specify a:

  • name: a unique name for the module.
  • loader: a loader that knows how to load the desired module. Loaders are an extension point for loading different types of modules. We provide loaders for use with modules written in native C, Node.js, Java, and .NET. The Hello World sample only uses the native C loader because all the modules in this sample are dynamic libraries written in C. For more information about how to use IoT Edge modules written in different languages, see the Node.js, Java, or .NET samples.

    • name: name of the loader used to load the module.
    • entrypoint: the path to the library containing the module. On Linux this library is a .so file, on Windows this library is a .dll file. The entry point is specific to the type of loader being used. The Node.js loader's entry point is a .js file. The Java loader's entry point is a classpath plus a class name. The .NET loader's entry point is an assembly name plus a class name.
  • args: any configuration information the module needs.

The following code shows the JSON used to declare all the IoT Edge modules for the Hello World sample on Linux. Whether a module requires any arguments depends on the design of the module. In this example, the logger module takes an argument that is the path to the output file and the hello_world module has no arguments.

"modules" :
[
    {
        "name" : "logger",
        "loader": {
          "name": "native",
          "entrypoint": {
            "module.path": "./modules/logger/liblogger.so"
        }
        },
        "args" : {"filename":"log.txt"}
    },
    {
        "name" : "hello_world",
        "loader": {
          "name": "native",
          "entrypoint": {
            "module.path": "./modules/hello_world/libhello_world.so"
        }
        },
        "args" : null
    }
]

The JSON file also contains the links between the modules that are passed to the broker. A link has two properties:

  • source: a module name from the modules section, or "*".
  • sink: a module name from the modules section.

Each link defines a message route and direction. Messages from module source are delivered to the module sink. The source may be set to "*", indicating that messages from any module are received by sink.

The following code shows the JSON used to configure links between the modules used in the hello_world sample on Linux. Every message produced by the hello_world module is consumed by the logger module.

"links":
[
    {
        "source": "hello_world",
        "sink": "logger"
    }
]

Hello_world module message publishing

You can find the code used by the hello_world module to publish messages in the 'hello_world.c' file. The following snippet shows an amended version of the code with comments added and some error handling code removed for legibility:

int helloWorldThread(void *param)
{
    // create data structures used in function.
    HELLOWORLD_HANDLE_DATA* handleData = param;
    MESSAGE_CONFIG msgConfig;
    MAP_HANDLE propertiesMap = Map_Create(NULL);

    // add a property named "helloWorld" with a value of "from Azure IoT
    // Gateway SDK simple sample!" to a set of message properties that
    // will be appended to the message before publishing it. 
    Map_AddOrUpdate(propertiesMap, "helloWorld", "from Azure IoT Gateway SDK simple sample!")

    // set the content for the message
    msgConfig.size = strlen(HELLOWORLD_MESSAGE);
    msgConfig.source = HELLOWORLD_MESSAGE;

    // set the properties for the message
    msgConfig.sourceProperties = propertiesMap;

    // create a message based on the msgConfig structure
    MESSAGE_HANDLE helloWorldMessage = Message_Create(&msgConfig);

    while (1)
    {
        if (handleData->stopThread)
        {
            (void)Unlock(handleData->lockHandle);
            break; /*gets out of the thread*/
        }
        else
        {
            // publish the message to the broker
            (void)Broker_Publish(handleData->brokerHandle, helloWorldMessage);
            (void)Unlock(handleData->lockHandle);
        }

        (void)ThreadAPI_Sleep(5000); /*every 5 seconds*/
    }

    Message_Destroy(helloWorldMessage);

    return 0;
}

Hello_world module message processing

The hello_world module never processes messages that other IoT Edge modules publish to the broker. Therefore, the implementation of the message callback in the hello_world module is a no-op function.

static void HelloWorld_Receive(MODULE_HANDLE moduleHandle, MESSAGE_HANDLE messageHandle)
{
    /* No action, HelloWorld is not interested in any messages. */
}

Logger module message publishing and processing

The logger module receives messages from the broker and writes them to a file. It never publishes any messages. Therefore, the code of the logger module never calls the Broker_Publish function.

The Logger_Recieve function in the logger.c file is the callback the broker invokes to deliver messages to the logger module. The following snippet shows an amended version with comments added and some error handling code removed for legibility:

static void Logger_Receive(MODULE_HANDLE moduleHandle, MESSAGE_HANDLE messageHandle)
{

    time_t temp = time(NULL);
    struct tm* t = localtime(&temp);
    char timetemp[80] = { 0 };

    // Get the message properties from the message
    CONSTMAP_HANDLE originalProperties = Message_GetProperties(messageHandle); 
    MAP_HANDLE propertiesAsMap = ConstMap_CloneWriteable(originalProperties);

    // Convert the collection of properties into a JSON string
    STRING_HANDLE jsonProperties = Map_ToJSON(propertiesAsMap);

    //  base64 encode the message content
    const CONSTBUFFER * content = Message_GetContent(messageHandle);
    STRING_HANDLE contentAsJSON = Base64_Encode_Bytes(content->buffer, content->size);

    // Start the construction of the final string to be logged by adding
    // the timestamp
    STRING_HANDLE jsonToBeAppended = STRING_construct(",{\"time\":\"");
    STRING_concat(jsonToBeAppended, timetemp);

    // Add the message properties
    STRING_concat(jsonToBeAppended, "\",\"properties\":"); 
    STRING_concat_with_STRING(jsonToBeAppended, jsonProperties);

    // Add the content
    STRING_concat(jsonToBeAppended, ",\"content\":\"");
    STRING_concat_with_STRING(jsonToBeAppended, contentAsJSON);
    STRING_concat(jsonToBeAppended, "\"}]");

    // Write the formatted string
    LOGGER_HANDLE_DATA *handleData = (LOGGER_HANDLE_DATA *)moduleHandle;
    addJSONString(handleData->fout, STRING_c_str(jsonToBeAppended);
}

Next steps

To learn about how to use the Azure IoT Edge, see the following articles: