Upload files from your device to the cloud with Azure IoT Hub (Node.js)

Article
03/16/2023

This article demonstrates how to file upload capabilities of IoT Hub upload a file to Azure blob storage, using Node.js.

The Send telemetry from a device to an IoT hub quickstart and Send cloud-to-device messages with IoT Hub articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The Configure Message Routing with IoT Hub tutorial shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example:

Videos
Large files that contain images
Vibration data sampled at high frequency
Some form of pre-processed data.

These files are typically batch processed in the cloud, using tools such as Azure Data Factory or the Hadoop stack. When you need to upland files from a device, you can still use the security and reliability of IoT Hub. This article shows you how.

At the end of this article, you run two Node.js console apps:

FileUpload.js, which uploads a file to storage using a SAS URI provided by your IoT hub.
FileUploadNotification.js, which receives file upload notifications from your IoT hub.

Note

IoT Hub supports many device platforms and languages (including C, Java, Python, and JavaScript) through Azure IoT device SDKs. Refer to the Azure IoT Developer Center to learn how to connect your device to Azure IoT Hub.

Important

File upload functionality on devices that use X.509 certificate authority (CA) authentication is in public preview, and preview mode must be enabled. It is generally available on devices that use X.509 thumbprint authentication or X.509 certificate attestation with Azure Device Provisioning Service. To learn more about X.509 authentication with IoT Hub, see Supported X.509 certificates.

Prerequisites

An IoT hub. Create one with the CLI or the Azure portal.
A registered device. Register one in the Azure portal.
Node.js version 10.0.x or later. The LTS version is recommended. You can download Node.js from nodejs.org.
Port 8883 should be open in your firewall. The device sample in this article uses MQTT protocol, which communicates over port 8883. This port may be blocked in some corporate and educational network environments. For more information and ways to work around this issue, see Connecting to IoT Hub (MQTT).

Associate an Azure Storage account to IoT Hub

To upload files from a device, you must have an Azure Storage account and Azure Blob Storage container associated with your IoT hub. Once you associate the storage account and container with your IoT hub, your IoT hub can provide the elements of a SAS URI when requested by a device. The device can then use these elements to construct the SAS URI that it uses to authenticate with Azure Storage and upload files to the blob container.

To associate an Azure Storage account with your IoT hub:

Under Hub settings, select File upload on the left-pane of your IoT hub.
On the File upload pane, select Azure Storage Container. For this article, it's recommended that your storage account and IoT Hub be located in the same region.
- If you already have a storage account you want to use, select it from the list.
- To create a new storage account, select +Storage account. Provide a name for the storage account and make sure the Location is set to the same region as your IoT hub, then select OK. The new account is created in the same resource group as your IoT hub. When the deployment completes, select the storage account from the list.
After you select the storage account, the Containers pane opens.
On the Containers pane, select the blob container.
- If you already have a blob container you want to use, select it from the list and click Select.
- To create a new blob container, select + Container. Provide a name for the new container. For the purposes of this article, you can leave all other fields at their default. Select Create. When the deployment completes, select the container from the list and click Select.
Back on the File upload pane, make sure that file notifications are set to On. You can leave all other settings at their defaults. Select Save and wait for the settings to complete before moving on to the next section.

For more detailed instructions on how to create an Azure Storage account, see Create a storage account. For more detailed instructions on how to associate a storage account and blob container with an IoT hub, see Configure file uploads using the Azure portal.

Upload a file from a device app

In this section, you create a device app to upload a file to IoT hub. The code is based on code available in the upload_to_blob_advanced.js sample in the Azure IoT Node.js SDK device samples.

Create an empty folder called fileupload. In the fileupload folder, create a package.json file using the following command at your command prompt. Accept all the defaults:
```
npm init
```
At your command prompt in the fileupload folder, run the following command to install the azure-iot-device Device SDK, the azure-iot-device-mqtt, and the @azure/storage-blob packages:
```
npm install azure-iot-device azure-iot-device-mqtt @azure/storage-blob --save
```

Using a text editor, create a FileUpload.js file in the fileupload folder, and copy the following code into it.

'use strict';

const Client = require('azure-iot-device').Client;
const Protocol = require('azure-iot-device-mqtt').Mqtt;
const errors = require('azure-iot-common').errors;
const path = require('path');

const {
  AnonymousCredential,
  BlockBlobClient,
  newPipeline
} = require('@azure/storage-blob');

// make sure you set these environment variables prior to running the sample.
const deviceConnectionString = process.env.DEVICE_CONNECTION_STRING;
const localFilePath = process.env.PATH_TO_FILE;
const storageBlobName = path.basename(localFilePath);

async function uploadToBlob(localFilePath, client) {
  const blobInfo = await client.getBlobSharedAccessSignature(storageBlobName);
  if (!blobInfo) {
    throw new errors.ArgumentError('Invalid upload parameters');
  }

  const pipeline = newPipeline(new AnonymousCredential(), {
    retryOptions: { maxTries: 4 },
    telemetry: { value: 'HighLevelSample V1.0.0' }, // Customized telemetry string
    keepAliveOptions: { enable: false }
  });

  // Construct the blob URL to construct the blob client for file uploads
  const { hostName, containerName, blobName, sasToken } = blobInfo;
  const blobUrl = `https://${hostName}/${containerName}/${blobName}${sasToken}`;

  // Create the BlockBlobClient for file upload to the Blob Storage Blob
  const blobClient = new BlockBlobClient(blobUrl, pipeline);

  // Setup blank status notification arguments to be filled in on success/failure
  let isSuccess;
  let statusCode;
  let statusDescription;

  try {
    const uploadStatus = await blobClient.uploadFile(localFilePath);
    console.log('uploadStreamToBlockBlob success');

    // Save successful status notification arguments
    isSuccess = true;
    statusCode = uploadStatus._response.status;
    statusDescription = uploadStatus._response.bodyAsText;

    // Notify IoT Hub of upload to blob status (success)
    console.log('notifyBlobUploadStatus success');
  }
  catch (err) {
    isSuccess = false;
    statusCode = err.code;
    statusDescription = err.message;

    console.log('notifyBlobUploadStatus failed');
    console.log(err);
  }

  await client.notifyBlobUploadStatus(blobInfo.correlationId, isSuccess, statusCode, statusDescription);
}

// Create a client device from the connection string and upload the local file to blob storage.
const deviceClient = Client.fromConnectionString(deviceConnectionString, Protocol);
uploadToBlob(localFilePath, deviceClient)
  .catch((err) => {
    console.log(err);
  })
  .finally(() => {
    process.exit();
  });

Save and close the FileUpload.js file.
Copy an image file to the fileupload folder and give it a name such as myimage.png.
Add environment variables for your device connection string and the path to the file that you want to upload. You got the device connection string when you registered a device in the IoT hub.
- For Windows:
```
set DEVICE_CONNECTION_STRING={your device connection string}
set PATH_TO_FILE={your image filepath}
```
- For Linux/Bash:
```
export DEVICE_CONNECTION_STRING="{your device connection string}"
export PATH_TO_FILE="{your image filepath}"
```

Get the IoT hub connection string

In this article, you create a backend service to receive file upload notification messages from the IoT hub you created. To receive file upload notification messages, your service needs the service connect permission. By default, every IoT Hub is created with a shared access policy named service that grants this permission.

To get the IoT Hub connection string for the service policy, follow these steps:

In the Azure portal, select Resource groups. Select the resource group where your hub is located, and then select your hub from the list of resources.
On the left-side pane of your IoT hub, select Shared access policies.
From the list of policies, select the service policy.
Copy the Primary connection string and save the value.

For more information about IoT Hub shared access policies and permissions, see Access control and permissions.

Receive a file upload notification

In this section, you create a Node.js console app that receives file upload notification messages from IoT Hub.

Create an empty folder called fileuploadnotification. In the fileuploadnotification folder, create a package.json file using the following command at your command prompt. Accept all the defaults:
```
npm init
```
At your command prompt in the fileuploadnotification folder, run the following command to install the azure-iothub SDK package:
```
npm install azure-iothub --save
```
Using a text editor, create a FileUploadNotification.js file in the fileuploadnotification folder.
Add the following require statements at the start of the FileUploadNotification.js file:
```
'use strict';

const Client = require('azure-iothub').Client;
```

Read the connection string for your IoT hub from the environment:

const connectionString = process.env.IOT_HUB_CONNECTION_STRING;

Add the following code to create a service client from the connection string:
```
const serviceClient = Client.fromConnectionString(connectionString);
```

Open the client and use the getFileNotificationReceiver function to receive status updates.

serviceClient.open(function (err) {
  if (err) {
    console.error('Could not connect: ' + err.message);
  } else {
    console.log('Service client connected');
    serviceClient.getFileNotificationReceiver(function receiveFileUploadNotification(err, receiver){
      if (err) {
        console.error('error getting the file notification receiver: ' + err.toString());
      } else {
        receiver.on('message', function (msg) {
          console.log('File upload from device:')
          console.log(msg.getData().toString('utf-8'));
          receiver.complete(msg, function (err) {
            if (err) {
              console.error('Could not finish the upload: ' + err.message);
            } else {
              console.log('Upload complete');
            }
          });
        });
      }
    });
  }
});

Note

If you want to receive disconnect notifications while you are listening to file upload notifications, you need to register 'error' by using receiver.on. To continue to receive file upload notifications, you need to reconect to IoT Hub by using the serviceClient.open method.

Save and close the FileUploadNotification.js file.
Add an environment variable for your IoT Hub connection string. You copied this string previously in Get the IoT hub connection string.
- For Windows:
```
set IOT_HUB_CONNECTION_STRING={your iot hub connection string}
```
- For Linux/Bash:
```
export IOT_HUB_CONNECTION_STRING="{your iot hub connection string}"
```

Run the applications

Now you're ready to run the applications.

At a command prompt in the fileuploadnotification folder, run the following command:

node FileUploadNotification.js

At a command prompt in the fileupload folder, run the following command:

node FileUpload.js

The following output is from the FileUpload app after the upload has completed:

uploadStreamToBlockBlob success
notifyBlobUploadStatus success

The following sample output is from the FileUploadNotification app after the upload has completed:

Service client connected
File upload from device:
{"deviceId":"myDeviceId","blobUri":"https://{your storage account name}.blob.core.windows.net/device-upload-container/myDeviceId/image.png","blobName":"myDeviceId/image.png","lastUpdatedTime":"2021-07-23T23:27:06+00:00","blobSizeInBytes":26214,"enqueuedTimeUtc":"2021-07-23T23:27:07.2580791Z"}

Verify the file upload

You can use the portal to view the uploaded file in the storage container you configured:

Navigate to your storage account in Azure portal.
On the left pane of your storage account, select Containers.
Select the container you uploaded the file to.
Select the folder named after your device.
Select the blob that you uploaded your file to. In this article, it's the blob with the same name as your file.
View the blob properties on the page that opens. You can select Download to download the file and view its contents locally.

Next steps

In this article, you learned how to use the file upload feature of IoT Hub to simplify file uploads from devices. You can continue to explore this feature with the following articles: