Modern standby Wi-Fi-connected scenarios

The goal of Wi-Fi-connected testing is to identify activities that should not be happening during modern standby and resolve them. The activities might be a result of network activities, real-time notifications, or unexpected hardware interrupts.

After the system can reliably enter and exit modern standby and can achieve good results in modern standby with airplane mode on, you can validate modern standby system behaviors with Wi-Fi connected.

To prepare for WiFi-connected testing, the following two prerequisites must be met:

  • The system has a reliable Wi-Fi connection. The best way to assess this is to test the Wi-Fi connection during active use to make sure that the connection is stable and signal/speed is reasonably good.
  • The Wi-Fi device is compatible with the NDIS 6.3 feature to support wake on LAN (WoL) patterns, protocol offloads, and D0 packet coalescing. These capabilities are required so that the SoC can enter low-power states while the Wi-Fi device maintains connectivity.

The list of Wi-Fi-connected scenarios includes connectivity to communications apps (such as Live, Lync, and Skype) and downloads from the Internet. These scenarios should be tested serially to focus on issues that might fix similar issues in other scenarios. The Windows Hardware Certification Kit (HCK) includes tests that help validate basic NDIS 6.3 functionality for modern standby. There are individual tests for each type of network adapter, including Wi-Fi, CDMA MBB, GSM MBB, and wired LAN. Please see the test details information for each test at Device.Network Testing.

We recommend that each network adapter in the system pass the following tests before you proceed to system-level connectivity testing for modern standby:

  • WLAN Connected Standby End to End - Basic
  • Win.MBN.CDMA.TestCSConnectivity
  • Win.MBN.GSM.TestCSConnectivity
  • LAN CS Test - IPv4 Basic

Modern standby push mail

Testing the push mail scenario ensures that the system can receive emails via the Mail app while the system is in modern standby. This allows users to receive the most up-to-date information immediately after the system resumes.

We recommend that you create a Microsoft account for testing purposes. You can use this account to set up the system to receive emails through the Mail app. Be sure that the system is running on AC power, then put it into modern standby. While the system is in modern standby, send an email to the account a few times. Wake up the system from modern standby and verify that all the emails are received. If Mail is configured to be on the lock screen, you should also see the badge updates occur just as the screen is powered on.

  • Test area: Push mail during modern standby.
  • Purpose: Ensure that push mail is functional during modern standby.
  • System configuration:
    • Factory image is installed on the system.
    • All drivers are loaded in Device Manager.
    • Factory Microsoft Store apps are installed.
    • System is running on AC power.
    • System is connected to a Wi-Fi access point (AP) that has Internet connectivity.
    • Mail app is configured with a test Microsoft account.
    • In Background Apps, Mail is set to On.
    • In Battery Usage:
      • Mail is set to Allow the app to run background tasks.
      • Mail is not set to Let Windows decide... or Reduce the work....
    • Email operation is first validated with the display on.
Test scenario Expected result Troubleshooting notes

The system can receive mail notifications while in modern standby.

If the Mail app is configured to be on the lock screen, updates should be seen on the lock screen immediately on resume from modern standby. Ensure all emails are received.

When a notification is not received, a trace is needed to diagnose the problem. For information about how to capture a trace and analyze the information, see Capture and View a WPA Trace for Modern Standby Diagnostics.

The Windows Hardware Certification Kit (HCK) includes modern standby tests that exercise entering and exiting modern standby on a system. Running the following tests will further validate the readiness of the system:

  • Modern standby exit latency test
  • Modern standby duration test
  • Modern standby stress with I/O

Modern standby instant messaging and video chat

Instant-message and video-chat notifications are important Wi-Fi-connected scenarios. The goal of testing this scenario is to ensure users can receive instant messages and video calls in real-time while the system is in modern standby.

You can use the test Microsoft account to configure the system with a Lync and Skype account. To test the instant messaging scenario, put the system in modern standby, then send an instant message to the test account. You should observe that the system powers on the display when an instant message is received and shows an instant message toast notification on the lock screen.

After you verify that instant message notifications are working, try to make a video call and an audio conference call. When the system is in modern standby, make a video call to the Skype test account. You should observe that the system powers on the display and shows a ringing toast notification at the upper right hand corner. If you do not pick up the call and let it finish ringing, the system will immediately go back into modern standby. Repeat this testing by making an audio conference call to the Lync test account.

The messaging and video/audio call experiences are exactly the same for both Lync and Skype. When the system wakes from modern standby, make sure that all notifications are received as expected. When an instant message or a call is missed, the icon on the lock screen will show the number of missed messages and calls.

The instant messaging and video chat scenarios are enumerated in the following table. All modern standby systems should be tested for expected behavior for each scenario that is listed in the table.

  • Test area: Instant messaging (IM) and video chat during modern standby.
  • Purpose: Ensure that real-time incoming functionality is operational during modern standby.
  • System configuration:
    • Factory image installed on the system.
    • All drivers loaded in Device Manager.
    • Factory Microsoft Store apps are installed.
    • System is running on battery power.
    • System is connected to a Wi-Fi AP with Internet connectivity.
    • Skype app configured with test Microsoft account.
    • Skype IM and video chat are validated to work with the screen turned on (outside of modern standby).
Test scenario Expected result Troubleshooting notes

System can receive IM notifications during modern standby.

When the system wakes from modern standby, check that all IM messages have been received and that the badge on the lock screen is updated appropriately.

If a notification is not received, a trace is needed to diagnose the problem. See Capture and View a WPA Trace for Modern Standby Diagnostics for information on how to capture a trace and analyze the information.

System can receive video call and audio conference call notifications during modern standby.

Ensure that all calls received during modern standby are accompanied by notifications that do the following:

  • Turn on the display to show a toast in the upper right-hand corner.

If the call is missed, the icon on the lock screen will show the missed call when the system wakes from modern standby.

If a notification is not received, a trace is needed to diagnose the problem. See Capture and View a WPA Trace for Modern Standby Diagnostics for information on how to capture a trace and analyze the information.

Modern standby background download

Background download through a Wi-Fi connection is an important modern standby scenario to test. This scenario includes downloads from Windows Update for critical updates, and downloads of music and movies while the system stays in modern standby.

The goal of testing background download is to make sure that download is done in the background and does not continuously block the system from entering the lowest power state during modern standby. Note that background download is allowed to continuously block the system from using the lowest power state when the system is running on AC power. Testing background download on battery is required.

To test background downloads, you should always try to download a file that is large enough that the download will continue to progress after the initial 20-minute period. You should also ensure the system is running on battery (not connected to AC) to get the correct behavior since systems on AC have special download policies to allow downloads to continue uninterrupted.

One way to test background download is use the inbox Video app to kick off a large download (over 3 gigabytes), and then put the system into modern standby. Windows permits the download to have approximately 20 minutes to complete after the start of a modern standby session when on battery power. After the initial 20 minutes, the download is throttled to enable only four seconds of network time every 15 minutes.

The key observation in the test is to verify that the large download does not prevent the system from entering the lowest power state after the first 20 minutes of the modern standby session.

On resume from modern standby, you can run a SleepStudy report to view the time when the system was in the lowest power state, to determine whether the background download was blocking the system from entering low power. The modern standby session should extend for one hour or more to account for the first 20 minutes of the session where background download is permitted.

  • Test area: Background file download during modern standby.
  • Purpose: Ensure that a background download is performed at low-power operation during modern standby.
  • System configuration:
    • Factory image installed on the system.
    • All drivers loaded in Device Manager.
    • Factory Microsoft Store apps are installed.
    • System is running on battery power.
    • System is configured with a test Microsoft account for inbox Video app access.
Test scenario Expected result Troubleshooting notes

Background downloads

A large file download should not continuously prevent the system from entering the lowest power state after the first 20 minutes.

With the exception of Windows Update, which can be active for a long period of time due to downloads of critical/security updates, all other downloads should not be active beyond the initial 20 minutes of the modern standby session.

You can use SleepStudy to check whether the large file download prevented the system from entering the lowest power state for the entirety of the session.

Modern standby roaming connectivity

The most basic roaming connectivity scenario involves functional validation of network list offload (NLO). NLO is the capability of the Wi-Fi device to automatically roam between previously associated Wi-Fi networks, without the involvement of Windows running on the SoC.

NLO starts with Windows tracking the list of previously associated Wi-Fi networks. When the system transitions into modern standby, Windows communicates the list of previously associated Wi-Fi networks to the Wi-Fi device.

If connectivity to the associated Wi-Fi network is lost, the Wi-Fi device periodically searches for one of the previously associated Wi-Fi networks while the device operates in a very low power mode. The periodic scans for a previous network are performed without waking the SoC so that the SoC can stay in a low-power mode. When the Wi-Fi device detects a previously used network, it connects to the network and wakes the SoC. Windows completes the connection, including obtaining an IP address and re-establishing system connections, including Windows Notification Services (WNS).

Start testing Wi-Fi roaming by setting up two Wi-Fi networks (A and B) with public Internet connections. Each Wi-Fi network should have a different SSID and be sufficiently distant so that the system cannot see both networks at the same time. Manually move the system near each Wi-Fi access point and connect to each network for a few minutes with the screen on and validate that Internet connectivity is functional. After the system has been demonstrated to connect to each Wi-Fi network reliably, you can start testing Wi-Fi roaming in modern standby.

Modern standby Wi-Fi roaming

The most basic roaming connectivity scenario involves functional validation of network list offload (NLO). NLO is the capability of the Wi-Fi device to automatically roam between previously associated Wi-Fi networks, without the involvement of Windows running on the SoC.

NLO starts with Windows tracking the list of previously associated Wi-Fi networks. When the system transitions into modern standby, Windows communicates the list of previously associated Wi-Fi networks to the Wi-Fi device.

If connectivity to the associated Wi-Fi network is lost, the Wi-Fi device periodically searches for one of the previously associated Wi-Fi networks while the device operates in a very low power mode. The periodic scans for a previous network are performed without waking the SoC so that the SoC can stay in a low-power mode. When the Wi-Fi device detects a previously used network, it connects to the network and wakes the SoC. Windows completes the connection, including obtaining an IP address and re-establishing system connections, including Windows Notification Services (WNS).

Start testing Wi-Fi roaming by setting up two Wi-Fi networks (A and B) with public Internet connections. Each Wi-Fi network should have a different SSID and be sufficiently distant so that the system cannot see both networks at the same time. Manually move the system near each Wi-Fi access point and connect to each network for a few minutes with the screen on and validate that Internet connectivity is functional. After the system has been demonstrated to connect to each Wi-Fi network reliably, you can start testing Wi-Fi roaming in modern standby.

  • Test area:Wi-Fi roaming operation during modern standby.
  • Purpose:Ensure that the system will automatically connect to previously used Wi-Fi networks during modern standby as the user moves the system between home, work, and coffee shops.
  • System configuration:
    • Factory image is installed on the system.
    • All drivers are loaded in Device Manager.
    • Factory Microsoft Store apps are installed.
    • System is running on battery power.
    • Wi-Fi connectivity is validated to work by connecting to the Internet while the screen is turned on.
Test scenario Expected result Troubleshooting notes

Wi-Fi connectivity is removed and the system automatically falls over to MBB while in modern standby.

Connect to the Wi-Fi network with MBB powered on, place the system into modern standby, and test push mail and Skype.

Mail and Skype calls should be received.

Power down the Wi-Fi access point and wait a few minutes for the system to automatically connect to the cellular network. Test push mail and Skype.

Mail and Skype calls should be received.

Power up the Wi-Fi access point again and wait a few minutes for the system to re-connect to Wi-Fi. Test push mail and Skype.

Mail and Skype calls should be received.

Make sure you first test Wi-Fi roaming in insolation before testing MBB failover.

Ensure that the system has a working cellular data connection.

If any part of the test fails, first have the Wi-Fi and MBB vendors help you verify that the device firmware has both WoL pattern and NLO capability enabled.

The test engineer for Wi-Fi roaming should build a close working relationship with the Wi-Fi vendor's support staff. Errors in this test should first be discussed with the Wi-Fi firmware developer.

A common problem is that the Wi-Fi device generates spurious SoC wake events because of false-positive NLO network detection events. This condition appears in SleepStudy as extra Wi-Fi device active time—typically, much more active time than any other PDC activator (such as BI, Windows Update, or WNS).

Modern standby Wi-Fi to MBB failover

During modern standby, if Wi-Fi connectivity is lost, Windows automatically transitions to an available MBB connection. This behavior enables the system to remain constantly connected during modern standby. However, Windows will favor the cheaper, faster, and (typically) more power-efficient Wi-Fi connection over a cellular-based connection.

MBB and cellular connectivity are highly influenced by signal strength and proximity to a cellular tower. Make sure that the system under test has good cellular network connectivity before you test this scenario.

You should fully validate basic Wi-Fi roaming before you try to validate failover from Wi-Fi to MBB. The same Wi-Fi NLO technology that enables basic Wi-Fi roaming is critical to the operation of Wi-Fi to MBB failover.

The system under test should be configured to have an active cellular data plan and SIM card. The test setup should include a Wi-Fi access point with public Internet access. We recommend that you configure the system under test with a Microsoft account, and set up an additional device that is configured to initiate email and Skype calls.

Start the Wi-Fi to MBB failover testing by confirming that the system is connected to the Wi-Fi access point and has MBB access. You should validate that both network types are functional with the screen on. To do this, connect to Wi-Fi with the MBB radio off, and then connect to MBB with the Wi-Fi radio off. For each network connection, you should connect to the Internet and browse the web to verify IP connectivity.

Next, put the system in modern standby with both Wi-Fi and MBB radios turned on and the system connected to the Wi-Fi network. Use a Wi-Fi connection to test push mail and Skype during modern standby. While the system under test remains in modern standby, power down the Wi-Fi access point and wait a few minutes for the system to connect to the cellular network. After a few minutes, re-test push mail and Skype calls to validate that Windows automatically connected to the cellular network and re-established connectivity.

  • Test area:Wi-Fi to MBB transitions during modern standby.
  • Purpose:Ensure that the system automatically connects to Wi-Fi or MBB during modern standby.
  • System configuration:
    • Factory image installed on the system.
    • All drivers loaded in Device Manager.
    • Factory Microsoft Store apps are installed.
    • System is running on battery power.
    • Wi-Fi and MBB are validated to function correctly with the screen on and manually configured connections to each network.
Test scenario Expected result Troubleshooting notes

System can receive IM notifications during modern standby.

Ensure that all IMs received during modern standby are accompanied by sound notifications, while the display remains powered off. When the system wakes from modern standby, check that all IM messages have been received and that the badge on the lock screen is updated appropriately.

If a notification is not received, a trace is needed to diagnose the problem. See Capture and View a WPA Trace for Modern Standby Diagnosticsfor information on how to capture a trace and analyze the information.