Troubleshoot Azure Sphere issues

This topic provides troubleshooting steps for the following types of problems:

General device connection problems

A failure to connect to the device from a host machine can occur for many reasons and may trigger any of several error messages, depending on which tools or applications encounter it. The following error messages may indicate a failed connection:

  • An error occurred. Please check your device is connected and your PC has been configured correctly, then retry.
  • Could not connect to the device. Check if your device is connected to the PC. The device may be unresponsive if it is applying an Azure Sphere operating system update; Wait a few minutes and then retry. If this issue persists, try uninstalling and reinstalling the Azure Sphere SDK.
  • An unexpected problem occurred. Please try again; if the issue persists, please refer to aka.ms/azurespheresupport for troubleshooting suggestions and support.
  • Failed to retrieve device ID from attached device: 'Could not connect to the device; please ensure it is attached.'
  • Failed to establish communication with device after recovery.

Failure to connect may occur for any of these reasons:

  • The board is not connected by USB to the host machine.

  • The board is not powered.

  • The device is currently busy applying an Azure Sphere OS update. During OS updates, the device may fail to respond.

To diagnose and resolve the problem, start with these basic steps:

  1. Ensure that the device is connected by USB to the host machine.

  2. Wait a few minutes. If OS update is progress, the device should again become responsive when the update completes.

  3. If the device is connected, press the Reset button on the device or disconnect and reconnect the USB cable. Wait ten seconds or so for the device to restart, and then try the failed command again.

  4. On Windows, run the azsphere device rescan-attached command. This command power-cycles the FTDI ports of all attached devices and can sometimes help in connection issues.

If none of these steps solves the problem, see Windows-specific problems or Linux-specific problems to determine whether the issue might be unique to your platform.

Device is unresponsive

If the device appears "dead" or unresponsive, the problem could be a lack of power.

Make sure that the real-time clock (RTC) is powered, by either the main 3V3 power supply or a coin cell battery. If your dev kit is compatible with the MT3620 reference development board (RDB), see the MT3620 development board user guide for details. If you're using a different dev kit, see the manufacturer's documentation.

In addition, make sure that the board itself is properly configured. For example, Seeed MT3620 Development Kits are shipped from the factory with a jumper header across pins 2 and 3 of J3, which powers the clock from the main power supply. Check that the header has not been dislodged or removed.

USB errors

If you encounter USB errors:

  1. Plug the device into a different USB port. If the error occurred when the device was plugged into a USB hub, plug it into a port on the machine instead.

  2. If the problem recurs, plug the device into a powered USB hub. In some cases, USB ports do not provide adequate power for the board, so a powered hub may solve the problem.

Dropped connections or missing devices

If your computer can connect to the Azure Sphere device, but often drops the connection, you may have a conflict in the IP subnet.

Azure Sphere uses subnet 192.168.35.*. If you have other software that uses the same subnet, either disable that software or limit the range of IP addresses that it uses. Currently, you cannot change the range of IP addresses that Azure Sphere uses.

If the subnet is not the problem, you may have a problem related to the FTDI driver used on Windows. Driver issues can cause PC-to-device connections to disappear intermittently, or can prevent the azsphere command from finding all attached devices. Run the azsphere device rescan-attached command to power-cycle the FTDI ports of all attached devices.

Cannot apply device capabilities

If you receive errors when you try to apply a device capability to an Azure Sphere device, the problem might be that the OS on your device is out of date. This problem can occur if the device has been offline for an extended period, during which Microsoft updated its internal keys. The following errors are symptoms of this problem:

The azsphere device enable-development or azsphere device capability update command returns an error similar to the following:

error: The device did not accept the device capability configuration. Please check the Azure Sphere OS on your device is up-to-date using 'azsphere device show-deployment-status'.

The azsphere device capability update command returns an error similar to the following:

error: Could not apply device capability configuration to device.

To solve this problem:

After your device has updated to the latest OS, you should once again be able to download and apply capabilities.

If you cannot successfully perform the steps to connect the device to the internet:

Windows-specific problems

In addition to the issues listed in Device connection problems, failure to connect on Windows may occur for any of these reasons:

  • The Azure Sphere Device Communication Service has not yet started.
  • The TAP-Windows adapter is not installed or is not configured correctly.

To resolve the problem from Windows, start with these basic steps:

  1. Make sure the device is connected to the PC by USB and has power.

  2. Restart the Azure Sphere Device Communication Service. Press Start and type Services. Right-click the Azure Sphere Device Communication Service and select Restart.

  3. If the error recurs after you restart the service, use the View Network Connections control panel to check that the adapter exists and is configured to use IP address 192.168.35.1. To access this control panel, press Start and type "ncpa.cpl". Right-click on the Azure Sphere device to view its properties. If Azure Sphere is not visible or does not have the correct IP address, reinstall the Azure Sphere SDK.

  4. If the device is connected, press the Reset button on the device. Wait ten seconds or so for the device to restart, and then try the failed command again.

  5. If the error recurs, unplug the device from the USB port, plug it in again, and wait for it to restart.

If none of these steps solves the problem, try the additional solutions that follow.

Failure to create four USB serial converters

After you set up an MT3620 development board, you should see four USB serial converters in Device Manager. If you see fewer than four, you may have a problem with the FTDI driver.

Note

If this board has previously been used for RTApp development, you may see three converters instead of four. This is normal and does not represent an error.

If the FTDI driver is not correctly installed, the converters might appear in the wrong location, such as Other devices, or might not appear at all.

To resolve this problem:

  1. Open Device Manager by clicking Start and typing "Device Manager."

  2. Under Universal Serial Bus controllers, select "USB Serial Converter A." Right-click the name, select Uninstall Device, and delete the driver if given the option:

    Uninstall device and delete driver

    Repeat this step for "USB Serial Converter B" through "USB Serial Converter D."

  3. Unplug your development board from your PC and plug it back in again. "MSFT MT3620 Std Interface" should appear with a triangle warning icon, which indicates no driver is available.

  4. Right-click on one of the "MSFT MT3620 Std Interface" devices and select Update driver. Choose "Search automatically for updated driver software." Updating one should fix them all. You should now see four USB serial converters in the Universal Serial Bus controllers section. If all four converters do not appear, repeat this step for each converter.

Failure to install FTDI drivers

The FTDI drivers should be downloaded and installed automatically by Windows when your Azure Sphere device is first plugged in to your PC. If the drivers are installed correctly, you will see four USB Serial Converters listed under Universal Serial Bus controllers in Device Manager, as described in Set up your dev kit.

Windows 10, version 2004, does not search for the drivers. In this case, the drivers are not downloaded and installed automatically and you will see the following items listed in Device Manager:

MT3620 items listed in Device Manager

To install the drivers, manually download the setup executable from FTDI and run it.

For availability dates and build numbers of Windows 10 versions, see Windows 10 release information. This information can help you determine whether your version of Windows 10 is earlier or later than version 2004.

Check the TAP-Windows adapter configuration

Azure Sphere tools communicate with attached development boards by using an IP network over USB. This requires the TAP-Windows adapter from OpenVPN Technologies. The Azure Sphere SDK installation procedure installs this adapter on your PC if it is not already present.

However, if a different version of the TAP-Windows adapter is already installed, or if the Azure Sphere device is not connected to the first instance of the TAP-Windows adapter, the Azure Sphere tools may fail to connect to your device.

To determine whether the problem is related to the TAP adapter, first find out how many TAP adapters are installed on your PC, and then modify the installation if necessary.

To determine how many TAP adapters are installed on your PC:

  1. Open Control Panel.

  2. In Windows Settings, select Network & Internet, and then select Change Adapter Options. You should see only one TAP adapter, as in the screenshot:

    One TAP Adapter

    If you see more than one TAP adapter, or if you see only one TAP adapter but its name is not Azure Sphere, follow these steps to uninstall all TAP adapters and reinstall the SDK. If you see no TAP adapters, reinstall the SDK.

To uninstall the TAP adapters:

  1. Click Start and type "Device Manager".

  2. In Device Manager, open Network Adapters and select TAP-Windows adapter:

    Device Manager with TAP adapter

  3. Right-click TAP-Windows adapter and select Uninstall Device. In the dialog box, check Delete the driver software for this device, then click Uninstall.

  4. Open a command prompt as Administrator and run the following Powershell installer script:

     powershell -ExecutionPolicy RemoteSigned -File "%ProgramData%\Microsoft\Azure Sphere\TapDriverInstaller\TapDriverInstaller.ps1" Install
    
  5. If the installation succeeds, restart the Azure Sphere Device Communication Service:

    net stop AzureSphereDeviceCommunicationService

    net start AzureSphereDeviceCommunicationService

  6. Reinstall the Azure Sphere SDK.

Device does not respond

One or more of the following errors from an azsphere command may indicate that the Azure Sphere Device Communication Service failed to start:

  • warn: Device is not responding. Could not perform version check.
  • Device is not responding. Cannot get device ID.​
  • error: Could not connect to the Azure Sphere Device Communication Service. If this issue persists, try uninstalling and reinstalling the Azure Sphere SDK.​
  • error: The device is not responding. The device may be unresponsive if it is applying an Azure Sphere operating system update; please retry in a few minutes.

The service may fail to start after Windows update and in cases where one of the internal JSON settings files or config file has become corrupted.

Failure after Windows Update

These errors can occur after you've updated Windows on your PC. Sometimes Windows Update uninstalls the FTDI drivers required for the communication service.

To resolve the problem:

  1. Unplug the Azure Sphere device from USB and plug it in again. Upon replugging the device, the correct drivers should reinstall.
  2. If unplugging and replugging the device fails to fix the problem, uninstall and reinstall the Azure Sphere SDK.

JSON file

If you have not recently updated Windows, the cause of the error might be the restore.json file that is used for the service.

To resolve this problem:

  1. Save a copy of the following file:

    c:\windows\serviceprofiles\localservice\appdata\local\Azure Sphere Tools\restore.json

  2. Delete the file from its original location.

  3. Stop and then restart the Azure Sphere Device Communication Service:

    net stop AzureSphereDeviceCommunicationService

    net start AzureSphereDeviceCommunicationService

Corrupted config file

If an error is reported when you try to run a command, it can be caused because the corrupted config file is preventing your device from running correctly.

To resolve this problem, delete the corrupted config file located in .azsphere\config on Windows or ~/.azsphere/config on Linux.

Windows crashes when plugging or unplugging a device

The MT3620 developer board has a Future Technology Devices International (FTDI) FT4232HQ chip, which facilitates communication between the device and PC. The official FTDI driver, Combined Driver Model (CMD), contains two drivers: one provides access through D2XX API, and the other provides a Virtual Com Port (VCP) for the same device. Both drivers are installed by default if the FTDI chip has VCP mode enabled. This may cause Windows to crash when the chip is power cycled.

To resolve this problem, you can disable VCP mode for the FTDI chip. You'll need to use the FT_PROG tool to reprogram the FTDI chip's EEPROM.

  1. See FTDI FT_PROG programming tool to find out how to download and install this tool.

  2. Run FT_PROG and find your attached device, as described in FT_PROG GUI application.

  3. In the Device Tree view, expand the Hardware Specific section. You should see four ports.

    FTProg Hardware Specific four ports

  4. Select Port A and choose D2XX Direct instead of Virtual Com Port.

    Select port A and D2XX direct

  5. Repeat the previous step for Port C and Port D. Port B should be in the D2XX Direct mode already.

  6. Click the Program Devices icon (looks like a lightning bolt) to enter programming mode.

    click program devices

  7. Program the EEPROM by clicking the Program button and then wait for it to finish.

    program devices

  8. Unplug your device from the USB port, then replug it, to power-cycle the device and cause the change to take effect. In Windows Device Manager, the Ports (COM & LPT) section should now show three fewer COM ports. The number of Universal Serial Bus devices should remain the same.

Lost connection to non-Azure Sphere FTDI devices after enabling RTApp debug

Some Azure Sphere users have reported that they can no longer communicate with other attached, non-Azure Sphere FTDI devices after they use the azsphere device enable-development --enable-rt-core-debugging command to develop and debug RTApps from their host PC.

For example, if you have both an Azure Sphere device and a different FTDI device attached to your PC, you may see two Universal Serial Controllers named "USB Serial Converter B" devices in Windows Device Manager before running the command.

device manager with two serial converter B

After executing the command, both USB Serial Converter B devices disappear from the Universal Serial Bus controller section and two new devices appear in the Universal Serial Bus devices display in Device Manager.

device manager two USB devices

Cause

This problem occurs because the azsphere device enable-development --enable-rt-core-debugging command installs a new driver for Port B of the FTDI chip on the MT3620; the port then becomes MSFT MT3620 Std Interface. However, installation of this driver inadvertently changes the driver for Port B of the other non-Azure Sphere device. Because of a limitation in the underlying library, all FTDI devices with the same VID (0x0403) and PID (0x6011) will have their Port B driver replaced.

Solution

Follow these steps to manually revert the driver for any non-Azure Sphere devices to its previous version:

  1. In Device Manager, select the non-Azure Sphere device ("Another FTDI Quad GZ" in the example), then right-click and select Update driver.

  2. In Update Drivers, select "Browse my computer for driver software".

    browse my computer for drivers

  3. Select "Let me pick from a list of available drivers on my computer.

    let me pick

  4. Select "USB Serial Converter B" driver from the list, and then click Next.

    usb serial converter b

  5. Click Close in the confirmation window.

  6. Device Manager should show Port B for the other FTDI device as USB Serial Converter B, which indicates that it uses the official FTDI driver. The driver for the MT3620 remains MSFT MT3620 Std Interface.

    Port B has reverted

Additional information

  • If you plug in another new non-Azure Sphere FTDI device after running the azsphere device enable-development --enable-rt-core-debugging command, that device will be assigned the Azure Sphere MT3620's driver in the same manner. Repeat the steps above to revert the device to the official FTDI driver.

  • If you unplug and re-plug a non-Azure Sphere FTDI device after you return it to the official FTDI driver, the device will keep the official FTDI driver.

  • If you run the azsphere device enable-development --enable-rt-core-debugging command again after you revert the driver, the non-Azure Sphere FTDI device will once again have its driver replaced, and you'll need to follow the steps in Solution to revert to the official FTDI driver. This occurs regardless of whether the non-Azure Sphere FTDI device is attached to the PC when the azsphere device enable-development --enable-rt-core-debugging command is run.

Commands not recognized

If you see the following error when you enter an azsphere command, ensure that you're using PowerShell or a standard command prompt on Windows.

'azsphere' is not recognized as an internal or external command, operable program or batch file.

Installer hangs at 60 percent

The installer hangs at 60%, and you are told that the Device Communication Service has failed to start. This typically occurs when the TAP driver installation fails in a strange way, leaving the system in an indeterminate state.

Confirm that the issue is the TAP driver:

  1. Open the Windows Event Viewer to check the logs.

  2. Look in the Application log and the Azure Sphere Device Communication Service log for the following error message:

    "SerialSlipToTun.TunInterfaceSetupException: Error access tun registry settings ---> System.Collections.Generic.KeyNotFoundException: Tun tap device not found ---> System.Security.SecurityException: Requested registry access is not allowed."

    Screenshot of the Windows Event Viewer.

  3. When checking the Application log, filter the log to avoid seeing the many unrelated messages. On the Action tab, select Filter Current Log.

  4. Select the Error checkbox and select AzureSphereDeviceCommunicationService to list only error messages from the Azure Sphere Device Communication Service.

    Screenshot of the Filter Current Log dialog box.

  5. If you cannot find the error in either the Application log or the Azure Sphere Device Communication Service log, then this may not be a TAP driver issue.

To resolve the TAP driver issue, follow these steps:

  1. Go to the Network and Sharing Center and select Change adapter settings.

    Screenshot of the Network Sharing Center.

  2. In Network Connections under Azure Sphere, select TAP-WIndows Adapter V9 and open its properties.

    Screenshot of Network Connections.

  3. In Azure Sphere Properties, select Internet Protocol Version 4 (TCP/IPv4) and then select the Properties button to view the protocol settings.

    Screenshot of Azure Sphere Properties.

  4. Ensure that IP address is set to 192.168.35.1 and Subnet mask is set to 255.255.255.0.

  5. Try the installer again. If it still hangs, try resetting your network connections. To do a reset, go to Settings > Network & Internet > Status and select Network reset near the bottom of the page.

    Beware that resetting your network will reset all network settings.

Linux-specific problems

On Linux systems, you need to run the azsphere_connect.sh script each time you plug in the device or unplug/replug it.

If you have run the script, failure to connect may occur for any of these reasons:

  • The board is not connected by USB.

  • The board is not powered.

To resolve the problem, start with these basic steps:

  1. Ensure that the device is connected by USB.

  2. Run the azsphere_connect.sh script, if you haven't already done so.

  3. If the device is connected, press the Reset button on the device. Wait ten seconds or so for the device to restart, and then try the failed command again.

  4. If the error recurs, unplug the device from the USB port, plug it in again, and wait for it to restart. Then run the azsphere_connect.sh script.

Cannot compile or debug RTApps

If CMake reports errors finding the toolchains or openocd when you try to build or debug an RTApp, make sure that: