Device Update Agent not starting when device connected through Azure IOT Edge

Question

Hello,

I am integrating Device Update Agent v0.8.0 to my project based on SAME70 based microcontroller. I am facing an issue on starting the DU-agent when the device is connected through Azure IOT Edge. The DU-agent starts/works fine when the device is connected directly to IOT Hub. I checked with the Device Update - Public Preview as well and observed same behavior.

When the issue happens, on calling the function, nx_azure_iot_adu_agent_start, the function does not return. I tracked down the issue to this function - nx_azure_iot_hub_client_properties_request_response. In this function, after mqtt packet is sent, the thread goes to sleep, but does not return from sleep. I have provided the code snippet, where the execution gets stuck, below.

status = nx_azure_iot_publish_mqtt_packet(&(hub_client_ptr -> nx_azure_iot_hub_client_resource.resource_mqtt),
                                              packet_ptr, topic_length, NX_NULL, NX_AZURE_IOT_MQTT_QOS_0,
                                              wait_option);

   if ((thread_list.thread_received_message) == NX_NULL && wait_option)
    {
        tx_thread_sleep(wait_option);
    }

I would appreciate any guidance on the possible cause for this issue.

Thank you.

// ************************************************************************************************** //
Edit 1:
// ************************************************************************************************** //

The difference in the success and the failure case seems to be in the response of the MQTT message received by the Azure RTOS device.

Success case response: $iothub/twin/res/204/?$rid=1&$version=1238§JÌEÐfÈJ–¬ƒì.0ZLl8..[Võ)ô.!÷Ñs

Failure case response: $iothub/twin/res/204/?$rid=1.w.˜Æþú/Øtû.ìé.OSWÈÓN]|3GGŠ™.îá

In success case, the mqtt receive callback function correctly identifies that the received response is a device twin update response.
In the failure case, the mqtt receive callback function does not find the version number in the response and returns error.

// ************************************************************************************************** //
Edit 2:
// ************************************************************************************************** //
I checked the message received by IOT Hub in both the failure and success cases. I could not find any difference other than the time and sequence related parameters. I am posting the message received by the IOT Hub in the failure case. Replaced device-id with "yyy" and iot hub id with "xxx".

event:
  annotations:
    iothub-connection-device-id: yyy
    iothub-enqueuedtime: 1647634839123
    iothub-message-source: digitalTwinChangeEvents
    x-opt-enqueued-time: 1647634839174
    x-opt-offset: '60129544904'
    x-opt-sequence-number: 9486
  component: ''
  interface: ''
  module: ''
  origin: yyy
  payload:
  - op: replace
    path: /deviceUpdate/$metadata/agent/lastUpdateTime
    value: '2022-03-18T20:20:38.9998555Z'
  properties:
    application:
      deviceId: yyy
      hubName: xxx
      iothub-message-schema: digitalTwinChangeNotification
      opType: updateTwin
      operationTimestamp: '2022-03-18T20:20:38.9998555+00:00'
    system:
      content_encoding: utf-8
      content_type: application/json-patch+json
      correlation_id: 3d1246e3863
      user_id: xxx

// ************************************************************************************************** //
Edit 3:
// ************************************************************************************************** //

I found that the case of missing version number is the same with other reported properties as well, and not just in case of Device Update Agent. While connecting through IOT Edge, the response gets discarded. However, in case of other reported properties, the wait option is (5 * NX_IP_PERIODIC_RATE). So, even if the response is discarded, the execution does not stop. In case of Device Update Agent, the wait option is WAIT_FOREVER, and hence the execution stops for that thread.

Answer 1

Hi @Sulav Lal Shrestha We discussed it internally with Edge team and SDK team. Given Edge Hub supports offline operations, the behaviors is not quite same between Edge Hub and IoT Hub, such as: the version number is not in response from Edge Hub. Then our SDK team fixed the version check on device in this PR https://github.com/Azure/azure-sdk-for-c/pull/2162

Answer 2

Hi @Sulav Lal Shrestha , nx_azure_iot_hub_client_properties_request_repsonse sends out properties to IoT Hub and waits for acknowledgement from IoT Hub, then nx_azure_iot_hub_client_properties_process processes the acknowledgement and resumes the sending thread if the acknowledgement is correct, but if there is no version string, az_iot_hub_client_twin_parse_received_topic returns ERROR.

Hi @azure-iot-edge, could you help to check why there is no version string in response from IoT Edge?

static UINT nx_azure_iot_hub_client_properties_process(NX_AZURE_IOT_HUB_CLIENT *hub_client_ptr,  
                                                       NX_PACKET *packet_ptr,  
                                                       ULONG topic_offset,  
                                                       USHORT topic_length)  
{  
  
    /* This function is protected by MQTT mutex. */  
    if ((status = nx_azure_iot_hub_client_device_twin_parse(hub_client_ptr, packet_ptr,  
                                                            topic_offset, topic_length,  
                                                            &request_id, &version,  
                                                            &message_type, &response_status)))  
    {  
        return(status);  
    }  

    ......  
}  


static UINT nx_azure_iot_hub_client_device_twin_parse(NX_AZURE_IOT_HUB_CLIENT *hub_client_ptr,  
                                                      NX_PACKET *packet_ptr, ULONG topic_offset,  
                                                      USHORT topic_length, UINT *request_id_ptr,  
                                                      ULONG *version_ptr, UINT *message_type_ptr,  
                                                      UINT *status_ptr)  
{  
az_result core_result;  
az_span topic_span;  
az_iot_hub_client_twin_response out_twin_response;  
uint32_t request_id = 0;  
uint32_t version;  
  
    topic_span = az_span_create(&(packet_ptr -> nx_packet_prepend_ptr[topic_offset]), (INT)topic_length);  
    core_result = az_iot_hub_client_twin_parse_received_topic(&(hub_client_ptr -> iot_hub_client_core),  
                                                              topic_span, &out_twin_response);  
    if (az_result_failed(core_result))  
    {  
        return(NX_AZURE_IOT_SDK_CORE_ERROR);  
    }  
  
    ......  
  
}