Maximizing and Optimizing your mailbox move data throughput for on-premise to O365 and Exchange Online Migrations

Well, that is one heck of a title for a blog post, isn't it?

I’d sent this to one of my customers who is migrating from 2010 to O365/Exchange Online. They started to implement some of the recommendations, and quickly found that they work!....which means they saturated their internet connection with mailbox moves, and had to tweak down, so the rest of the company still had internet access.

This customer had been migrating test batches of users and had found they were only averaging a rate of 1-GB an hour for batches of 100 users. The following table from Exchange Online migration performance and best practices indicates that they could get much better migration velocity:

Migration Method  Office 365 user throttling  Office 365 migration-service throttling  Office 365 resource health-based throttling  Observed average throughput per hour and per client (if applicable)  
Hybrid Migration No Yes Yes 10-14 GB per on-premises Exchange 2013 or 2010 CAS (MRS Proxy) with 20 concurrent moves1

I gathered some resources from Microsoft’s internal and public guidance to come up with an initial action plan to identify where we could either improve the throughput for migrations, or identify environmental or configuration items that may be hampering migration performance.

Identify which servers you have enabled for MRS Proxy enabled:

Run the following command in the Shell from one of the Exchange 2013 Hybrid servers

Get-WebServicesVirtualDirectory | FL Identity,MRSProxyEnabled

Note which servers have the  MRSProxyEnabled parameter set to True

Another way to verify that the MRS Proxy endpoint is enabled is to use the Test-MigrationServerAvailability cmdlet to test the ability to communicate with the remote server that hosts the mailboxes that you want to move.

The following example tests the connection to a server in the corp.contoso.com forest.

$Credentials = Get-Credential

Test-MigrationServerAvailability -ExchangeRemoteMove -Autodiscover -EmailAddress administrator@corp.contoso.com -Credentials $Credentials

 

Once we identify which CAS are configured as MRS Proxy servers, we can examine the intervening network topology and devices to ensure that the MRS Proxy servers are published correctly. Issues can occur if, for example, the 2013 Hybrid servers are routing out via a load balancer that they may not be correctly configured for. Identifying the MRS Proxies would be an initial step in identifying potential issues that may impede migration performance.

  1. Another tool that can help identify if there are issues that are impeding migration throughput is the script that is published on Microsoft’s official Exchange EHLO Blog, Mailbox Migration Performance Analysis.  An another script that can help assist idenitifying issues is published on Microsoft’s Technet Gallery called MRS Explorer
  2. Yet another Network based performance tool is mentioned on Exchange Online migration performance and best practices:

 

Network performance factors

This section describes best practices for improving network performance during migration. The discussion is general because the biggest impact on network performance during migration is related to third-party hardware and Internet service providers (ISPs).

The Office 365 Network Analysis Tool is deployed to help analyze network-related issues prior to deploying Office 365 services:. Each of these instances is designed to test a particular region using test end-points in Office 365.

North America: https://na1-fasttrack.cloudapp.net

  1. There are also optimizations that can be made to the Exchange servers that are acting as MRS Proxies, to increase throughput.

For Exchange 2013 MRS Proxy enabled servers:

The following technique involves adding a configuration line to the MSExchangeMailboxReplication.exe.config file on the Exchange 2013 Server:

Increase the export buffer size. This reduces the number of migration calls, especially for larger mailboxes and reduces the time spent in network latency. You can increase the export buffer size by adding the ExportBufferSizeOverrideKB parameter in the

MSExchangeMailboxReplication.exe.config:

Example: ExportBufferSizeOverrideKB="7500"

<MRSProxyConfiguration

   MaxMRSConnections="100"

   DataImportTimeout="00:05:00" />

  ExportBufferSizeOverrideKB="7500"

The ExportBufferSizeOverrideKB value may not be present by default in the MSExchangeMailboxReplication.exe.config file. It is also recommended to change the DataImportTimeout value to 5 minutes as highlighted in the example above, as it is default set to 1 minute.

 

For Exchange 2010 MRS Proxy enabled CAS-role servers:

If the production Exchange 2010 CAS are being used as MRS Proxies:

All Exchange 2010 CAS that are in the load balanced namespace are required to be enabled for MRS Proxy. For example, if the MRS Proxy endpoint is published to the internet as https://webmail.contoso.com, and that namespace is loadbalanced, then all Exchange 2010 CAS that participate in that load balanced configuration would have to have the MRS Proxy capability enabled on them,

Several settings can reduce performance for mailbox moves. Editing the MSExchangeMailboxReplication.exe.config file, located on all CAS servers in the same folder where Exchange is installed: \Program Files\Microsoft\Exchange Server\V14\Bin\MSExchangeMailboxReplication.exe.config. and modifying the following parameters to higher values can help improve migration throughput if an Exchange 2010 CAS is enabled as an MRS Proxy:

MaxActiveMovesPerSourceMDB - This property specifies the total number of tasks MRS can perform that involve the mailbox database as a data source. Types of tasks include moving mailboxes located on the database, exporting mailbox data from mailboxes located on the database, and restoring mailbox data from the database. You can specify a value from 0 through 100. The default value is 5 concurrent tasks. Consider raising to 10

MaxActiveMovesPerTargetMDB - This property specifies the total number of tasks MRS can perform that involve the mailbox database as a data target. Types of tasks include moving mailboxes to the database, importing mailbox data into a mailbox located on the database, and restoring mailbox data to a mailbox located on the database. You can specify a value from 0 through 100 . The default value is 2 concurrent tasks. Consider raising to 10 or higher.

MaxActiveMovesPerSourceServer - This property specifies the total number of tasks MRS can perform that include the server as a data source. You can specify a value from 0 through 1000. The default value is 50 concurrent moves. I don’t see this as an issue, but mentioning it for thoroughness.

MaxActiveMovesPerTargetServer - This property specifies the total number of tasks MRS can perform that involve the server as a data target. You can specify a value from 0 through 1000. The default value is 5 concurrent moves. Consider raising to 10