Create and provision IoT Edge devices at scale with a TPM on Linux

Applies to: yes icon IoT Edge 1.1 Other versions: IoT Edge 1.2

Applies to: yes icon IoT Edge 1.2 Other versions: IoT Edge 1.1

This article provides instructions for autoprovisioning an Azure IoT Edge for Linux device by using a Trusted Platform Module (TPM). You can automatically provision IoT Edge devices with the Azure IoT Hub device provisioning service. If you're unfamiliar with the process of autoprovisioning, review the provisioning overview before you continue.

This article outlines two methodologies. Select your preference based on the architecture of your solution:

  • Autoprovision a Linux device with physical TPM hardware. An example is the Infineon OPTIGA™ TPM SLB 9670.
  • Autoprovision a Linux virtual machine (VM) with a simulated TPM running on a Windows development machine with Hyper-V enabled. We recommend using this methodology only as a testing scenario. A simulated TPM doesn't offer the same security as a physical TPM.

Instructions differ based on your methodology, so make sure you're on the correct tab going forward.

The tasks are as follows:

  1. Retrieve provisioning information for your TPM.
  2. Create an individual enrollment for your device in an instance of the IoT Hub device provisioning service.
  3. Install the IoT Edge runtime and connect the device to the IoT hub.

Prerequisites

Cloud resources

  • An active IoT hub
  • An instance of the IoT Hub device provisioning service in Azure, linked to your IoT hub

Device requirements

A physical Linux device to be the IoT Edge device.

Note

TPM 2.0 is required when you use TPM attestation with the device provisioning service.

You can only create individual, not group, device provisioning service enrollments when you use a TPM.

Set up your device

If you're using a physical Linux device with a TPM, there are no extra steps to set up your device.

You're ready to continue.

Retrieve provisioning information for your TPM

In this section, you build a tool that you can use to retrieve the registration ID and endorsement key for your TPM.

  1. Sign in to your device, and then follow the steps in Set up a Linux development environment to install and build the Azure IoT device SDK for C.

  2. Run the following commands to build the SDK tool that retrieves your device provisioning information for your TPM.

    cd azure-iot-sdk-c/cmake
    cmake -Duse_prov_client:BOOL=ON ..
    cd provisioning_client/tools/tpm_device_provision
    make
    sudo ./tpm_device_provision
    
  3. The output window displays the device's Registration ID and the Endorsement key. Copy these values for use later when you create an individual enrollment for your device in the device provisioning service.

Tip

If you don't want to use the SDK tool to retrieve the information, you need to find another way to obtain the provisioning information. The endorsement key, which is unique to each TPM chip, is obtained from the TPM chip manufacturer associated with it. You can derive a unique registration ID for your TPM device. For example, you can create an SHA-256 hash of the endorsement key.

After you have your registration ID and endorsement key, you're ready to continue.

Create a device provisioning service enrollment

Use your TPM's provisioning information to create an individual enrollment in the device provisioning service.

When you create an enrollment in the device provisioning service, you have the opportunity to declare an Initial Device Twin State. In the device twin, you can set tags to group devices by any metric used in your solution, like region, environment, location, or device type. These tags are used to create automatic deployments.

Tip

The steps in this article are for the Azure portal, but you can also create individual enrollments by using the Azure CLI. For more information, see az iot dps enrollment. As part of the CLI command, use the edge-enabled flag to specify that the enrollment is for an IoT Edge device.

  1. In the Azure portal, go to your instance of the IoT Hub device provisioning service.

  2. Under Settings, select Manage enrollments.

  3. Select Add individual enrollment, and then complete the following steps to configure the enrollment:

    1. For Mechanism, select TPM.

    2. Provide the Endorsement key and Registration ID that you copied from your VM or physical device.

    3. Provide an ID for your device if you want. If you don't provide a device ID, the registration ID is used.

    4. Select True to declare that your VM or physical device is an IoT Edge device.

    5. Choose the linked IoT hub that you want to connect your device to, or select Link to new IoT Hub. You can choose multiple hubs, and the device will be assigned to one of them according to the selected assignment policy.

    6. Add a tag value to the Initial Device Twin State if you want. You can use tags to target groups of devices for module deployment. For more information, see Deploy IoT Edge modules at scale.

    7. Select Save.

Now that an enrollment exists for this device, the IoT Edge runtime can automatically provision the device during installation.

Install IoT Edge

In this section, you prepare your Linux VM or physical device for IoT Edge. Then, you install IoT Edge.

First, run the following commands to add the package repository and then add the Microsoft package signing key to your list of trusted keys.

Installing can be done with a few commands. Open a terminal and run the following commands:

  • 20.04:

    wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
    sudo dpkg -i packages-microsoft-prod.deb
    rm packages-microsoft-prod.deb
    
  • 18.04:

    wget https://packages.microsoft.com/config/ubuntu/18.04/multiarch/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
    sudo dpkg -i packages-microsoft-prod.deb
    rm packages-microsoft-prod.deb
    

Note

Azure IoT Edge software packages are subject to the license terms located in each package (usr/share/doc/{package-name} or the LICENSE directory). Read the license terms prior to using a package. Your installation and use of a package constitutes your acceptance of these terms. If you don't agree with the license terms, don't use that package.

Install a container engine

Azure IoT Edge relies on an OCI-compatible container runtime. For production scenarios, we recommend that you use the Moby engine. The Moby engine is the only container engine officially supported with IoT Edge. Docker CE/EE container images are compatible with the Moby runtime.

Install the Moby engine.

sudo apt-get update; \
  sudo apt-get install moby-engine

Once the Moby engine is successfully installed, configure it to use local logging driver as the logging mechanism. To learn more about logging configuration, see Production Deployment Checklist.

  • Create or open the Docker daemon's config file at /etc/docker/daemon.json.

  • Set the default logging driver to the local logging driver as shown in the example below.

       {
          "log-driver": "local"
       }
    
  • Restart the container engine for the changes to take effect.

    Tip

    If you get errors when you install the Moby container engine, verify your Linux kernel for Moby compatibility. Some embedded device manufacturers ship device images that contain custom Linux kernels without the features required for container engine compatibility. Run the following command, which uses the check-config script provided by Moby, to check your kernel configuration:

    curl -ssl https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh -o check-config.sh
    chmod +x check-config.sh
    ./check-config.sh
    

    In the output of the script, check that all items under Generally Necessary and Network Drivers are enabled. If you're missing features, enable them by rebuilding your kernel from source and selecting the associated modules for inclusion in the appropriate kernel .config. Similarly, if you're using a kernel configuration generator like defconfig or menuconfig, find and enable the respective features and rebuild your kernel accordingly. After you've deployed your newly modified kernel, run the check-config script again to verify that all the required features were successfully enabled.

Install the IoT Edge runtime

The IoT Edge security daemon provides and maintains security standards on the IoT Edge device. The daemon starts on every boot and bootstraps the device by starting the rest of the IoT Edge runtime.

The steps in this section represent the typical process to install the latest version on a device that has internet connection. If you need to install a specific version, like a pre-release version, or need to install while offline, follow the Offline or specific version installation steps later in this article.

Install IoT Edge version 1.1.* along with the libiothsm-std package:

sudo apt-get update; \
  sudo apt-get install iotedge

Note

IoT Edge version 1.1 is the long-term support branch of IoT Edge. If you are running an older version, we recommend installing or updating to the latest patch as older versions are no longer supported.

The IoT Edge service provides and maintains security standards on the IoT Edge device. The service starts on every boot and bootstraps the device by starting the rest of the IoT Edge runtime.

The IoT identity service was introduced along with version 1.2 of IoT Edge. This service handles the identity provisioning and management for IoT Edge and for other device components that need to communicate with IoT Hub.

The steps in this section represent the typical process to install the latest version on a device that has internet connection. If you need to install a specific version, like a pre-release version, or need to install while offline, follow the Offline or specific version installation steps later in this article.

Note

The steps in this section show you how to install IoT Edge version 1.2.

If you already have an IoT Edge device running an older version and want to upgrade to 1.2, use the steps in Update the IoT Edge security daemon and runtime. Version 1.2 is sufficiently different from previous versions of IoT Edge that specific steps are necessary to upgrade.

Install the latest version of IoT Edge and the IoT identity service package:

sudo apt-get update; \
  sudo apt-get install aziot-edge defender-iot-micro-agent-edge

The defender-iot-micro-agent-edge package includes the Microsoft Defender for IoT security micro-agent that provides endpoint visibility into security posture management, vulnerabilities, threat detection, fleet management and more to help you secure your IoT Edge devices. It is recommended to install the micro agent with the Edge agent to enable security monitoring and hardening of your Edge devices. To learn more about Microsoft Defender for IoT, see What is Microsoft Defender for IoT for device builders.

Provision the device with its cloud identity

After the runtime is installed on your device, configure the device with the information it uses to connect to the device provisioning service and IoT Hub.

  1. Know your device provisioning service ID Scope and device Registration ID that were gathered previously.

  2. Open the configuration file on your IoT Edge device.

    sudo nano /etc/iotedge/config.yaml
    
  3. Find the provisioning configuration section of the file. Uncomment the lines for TPM provisioning, and make sure any other provisioning lines are commented out.

    The provisioning: line should have no preceding whitespace, and nested items should be indented by two spaces.

    # DPS TPM provisioning configuration
    provisioning:
      source: "dps"
      global_endpoint: "https://global.azure-devices-provisioning.net"
      scope_id: "SCOPE_ID_HERE"
      attestation:
        method: "tpm"
        registration_id: "REGISTRATION_ID_HERE"
    
    # always_reprovision_on_startup: true
    # dynamic_reprovisioning: false
    
  4. Update the values of scope_id and registration_id with your device provisioning service and device information. The scope_id value is the ID Scope from your device provisioning service instance's overview page.

  5. Optionally, use the always_reprovision_on_startup or dynamic_reprovisioning lines to configure your device's reprovisioning behavior. If a device is set to reprovision on startup, it will always attempt to provision with DPS first and then fall back to the provisioning backup if that fails. If a device is set to dynamically reprovision itself, IoT Edge (and all modules) will restart and reprovision if a reprovisioning event is detected, like if the device is moved from one IoT Hub to another. Specifically, IoT Edge checks for bad_credential or device_disabled errors from the SDK to detect the reprovision event. To trigger this event manually, disable the device in IoT Hub. For more information, see IoT Hub device reprovisioning concepts.

  6. Save and close the file.

  1. Know your device provisioning service ID Scope and device Registration ID that were gathered previously.

  2. Create a configuration file for your device based on a template file that's provided as part of the IoT Edge installation.

    sudo cp /etc/aziot/config.toml.edge.template /etc/aziot/config.toml
    
  3. Open the configuration file on the IoT Edge device.

    sudo nano /etc/aziot/config.toml
    
  4. Find the provisioning configurations section of the file. Uncomment the lines for TPM provisioning, and make sure any other provisioning lines are commented out.

    # DPS provisioning with TPM
    [provisioning]
    source = "dps"
    global_endpoint = "https://global.azure-devices-provisioning.net"
    id_scope = "SCOPE_ID_HERE"
    
    [provisioning.attestation]
    method = "tpm"
    registration_id = "REGISTRATION_ID_HERE"
    
    # auto_reprovisioning_mode = Dynamic
    
  5. Update the values of id_scope and registration_id with your device provisioning service and device information. The scope_id value is the ID Scope from your device provisioning service instance's overview page.

  6. Optionally, find the auto reprovisioning mode section of the file. Use the auto_reprovisioning_mode parameter to configure your device's reprovisioning behavior. Dynamic - Reprovision when the device detects that it may have been moved from one IoT Hub to another. This is the default. AlwaysOnStartup - Reprovision when the device is rebooted or a crash causes the daemon(s) to restart. OnErrorOnly - Never trigger device reprovisioning automatically. Each mode has an implicit device reprovisioning fallback if the device is unable to connect to IoT Hub during identity provisioning due to connectivity errors. For more information, see IoT Hub device reprovisioning concepts.

  7. Save and close the file.

Give IoT Edge access to the TPM

The IoT Edge runtime needs to access the TPM to automatically provision your device.

You can give TPM access to the IoT Edge runtime by overriding the systemd settings so that the iotedge service has root privileges. If you don't want to elevate the service privileges, you can also use the following steps to manually provide TPM access.

  1. Create a new rule that will give the IoT Edge runtime access to tpm0 and tpmrm0.

    sudo touch /etc/udev/rules.d/tpmaccess.rules
    
  2. Open the rules file.

    sudo nano /etc/udev/rules.d/tpmaccess.rules
    
  3. Copy the following access information into the rules file. The tpmrm0 might not be present on devices that use a kernel earlier than 4.12. Devices that don't have tpmrm0 will safely ignore that rule.

    # allow iotedge access to tpm0
    KERNEL=="tpm0", SUBSYSTEM=="tpm", OWNER="iotedge", MODE="0600"
    KERNEL=="tpmrm0", SUBSYSTEM=="tpmrm", OWNER="iotedge", MODE="0600"
    
  4. Save and exit the file.

  5. Trigger the udev system to evaluate the new rule.

    /bin/udevadm trigger --subsystem-match=tpm --subsystem-match=tpmrm
    
  6. Verify that the rule was successfully applied.

    ls -l /dev/tpm*
    

    Successful output appears as follows:

    crw------- 1 iotedge root 10, 224 Jul 20 16:27 /dev/tpm0
    crw------- 1 iotedge root 10, 224 Jul 20 16:27 /dev/tpmrm0
    

    If you don't see that the correct permissions have been applied, try rebooting your machine to refresh udev.

  7. Restart the IoT Edge runtime so that it picks up all the configuration changes that you made on the device.

    sudo systemctl restart iotedge
    

The IoT Edge runtime relies on a TPM service that brokers access to a device's TPM. This service needs to access the TPM to automatically provision your device.

You can give access to the TPM by overriding the systemd settings so that the aziottpm service has root privileges. If you don't want to elevate the service privileges, you can also use the following steps to manually provide TPM access.

  1. Create a new rule that will give the IoT Edge runtime access to tpm0 and tpmrm0.

    sudo touch /etc/udev/rules.d/tpmaccess.rules
    
  2. Open the rules file.

    sudo nano /etc/udev/rules.d/tpmaccess.rules
    
  3. Copy the following access information into the rules file. The tpmrm0 might not be present on devices that use a kernel earlier than 4.12. Devices that don't have tpmrm0 will safely ignore that rule.

    # allow aziottpm access to tpm0 and tpmrm0
    KERNEL=="tpm0", SUBSYSTEM=="tpm", OWNER="aziottpm", MODE="0660"
    KERNEL=="tpmrm0", SUBSYSTEM=="tpmrm", OWNER="aziottpm", MODE="0660"
    
  4. Save and exit the file.

  5. Trigger the udev system to evaluate the new rule.

    /bin/udevadm trigger --subsystem-match=tpm --subsystem-match=tpmrm
    
  6. Verify that the rule was successfully applied.

    ls -l /dev/tpm*
    

    Successful output appears as follows:

    crw-rw---- 1 root aziottpm 10, 224 Jul 20 16:27 /dev/tpm0
    crw-rw---- 1 root aziottpm 10, 224 Jul 20 16:27 /dev/tpmrm0
    

    If you don't see that the correct permissions have been applied, try rebooting your machine to refresh udev.

  7. Apply the configuration changes that you made on the device.

    sudo iotedge config apply
    

Verify successful installation

If you didn't already, restart the IoT Edge runtime so that it picks up all the configuration changes that you made on the device.

sudo systemctl restart iotedge

Check to see that the IoT Edge runtime is running.

sudo systemctl status iotedge

Examine daemon logs.

journalctl -u iotedge --no-pager --no-full

If you see provisioning errors, it might be that the configuration changes haven't taken effect yet. Try restarting the IoT Edge daemon again.

sudo systemctl daemon-reload

Or, try restarting your VM to see if the changes take effect on a fresh start.

If you didn't already, apply the configuration changes that you made on the device.

sudo iotedge config apply

Check to see that the IoT Edge runtime is running.

sudo iotedge system status

Examine daemon logs.

sudo iotedge system logs

If you see provisioning errors, it might be that the configuration changes haven't taken effect yet. Try restarting the IoT Edge daemon.

sudo systemctl daemon-reload

Or, try restarting your VM to see if the changes take effect on a fresh start.

If the runtime started successfully, you can go into your IoT hub and see that your new device was automatically provisioned. Now your device is ready to run IoT Edge modules.

List running modules.

iotedge list

You can verify that the individual enrollment that you created in the device provisioning service was used. Go to your device provisioning service instance in the Azure portal. Open the enrollment details for the individual enrollment that you created. Notice that the status of the enrollment is assigned and the device ID is listed.

Next steps

The device provisioning service enrollment process lets you set the device ID and device twin tags at the same time as you provision the new device. You can use those values to target individual devices or groups of devices by using automatic device management.

Learn how to deploy and monitor IoT Edge modules at scale by using the Azure portal or the Azure CLI.