question

GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 asked ·

Cannot move mailbox to a new Database - Exchange 2016 on Prem

Not sure how to fix this, had no problem moving a mailbox to a new database before, but trying to move a mailbox to a new database and it is stuck at syncing and when I check the status, I see the following error - StalledDueToTarget_MdbReplication. I have checked both the databases on the DAG and there are no health issues and copy queue length is 0. I have tried to move 2 separate mailboxes on separate databases and the same thing is happening. I have also tried from both the Web GUI and powershell and they both seem to stall at the same 37%. I have also verified that the Migration Mailbox is listed when checking for the Arbitration mailbox. I am not sure what else to check, but seems I cannot migrate any mailboxes at this point. Anyone have any suggestions on what could be causing this issue?

Exchange 2016 - 2 DAG groups - 4 Servers - Hybrid mode with all mailboxes on-prem. Tried to move a mailbox from each DAG and have the same problem on both of them.

Thanks,
Gavin

office-exchange-server-administrationoffice-exchange-hybrid-itpro
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 answered ·

So it appears that the problem was related to the other issue you were helping me with regarding the missing Self Signed Cert, after I fixed that issue, the replication errors went away as well.

[PS] C:\WINDOWS\system32>Test-ReplicationHealth -Identity SERVER1

Server Check Result Error


SERVER1 ClusterService Passed
SERVER1 ReplayService Passed
SERVER1 ActiveManager Passed
SERVER1 TasksRpcListener Passed
SERVER1 TcpListener Passed
SERVER1 ServerLocatorService Passed
SERVER1 DagMembersUp Passed
SERVER1 MonitoringService Passed
SERVER1 ClusterNetwork Passed
SERVER1 QuorumGroup Passed
SERVER1 FileShareQuorum Passed
SERVER1 DatabaseRedundancy Passed
SERVER1 DatabaseAvailability Passed
SERVER1 DBCopySuspended Passed
SERVER1 DBCopyFailed Passed
SERVER1 DBInitializing Passed
SERVER1 DBDisconnected Passed
SERVER1 DBLogCopyKeepingUp Passed
SERVER1 DBLogReplayKeepingUp Passed

·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

LucasLiu-MSFT avatar image
0 Votes"
LucasLiu-MSFT answered ·

Hi @GavinRoss-3568 ,
1.According to my research on “Stalledduetotarget_mdbreplication”, this value is also returned from Data Guarantee API on checking the replication health of the target database copies if they are a member of DAG and have database copies. We might get this message if the MRS service is waiting to get this information from the target server about the replication status of the database copies. Please run the following commands to check the status of database copy and the status of replication.

 Get-MailboxDatabaseCopyStatus -Identity <DAG> | Format-List
 Test-ReplicationHealth

2.Please try to restart the mailbox replication service, then clear all move requests and create new move requests.
3.Please run the following command to modifying the priority to highest:

 New-MoveRequest -Identity <> -TargetDatabase “DB Name” -BatchName <> -Priority Highest

For more information : Exchange 2016 Migration Status

4.In addition, based on the research of similar cases, I found that some users use the following methods to disable MRS Resource Health, after which they can successfully migrate their mailboxes. It should be noted that after modifying the registry, we need to restart the computer for the modification to take effect.

Regedit -->find the key "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchange ResourceHealth\MRS" --> set "MRS" REG_DWORD value from 1 to 0 --> Restart Exchange Mailbox Replication service

After migration is completed, we can revert back MRS value from 0 to 1 in regedit.
In order to prevent the impact of incorrect modification, you could also back up your Regedit in advance: How to back up and restore the registry in Windows


If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.



· 4 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for the info, but I have already run all these commands. I run a daily health check on all databases/replication/DAG and there are no issues. I have already run the powershell command New-MoveRequest -Identity <> -TargetDatabase “DB Name” -BatchName <> -Priority Highest and that has not worked either. My concern with changing the registry key, which if works, would have to be done each time we want to move a database. I am hoping that there would be a permanent solution to this problem.

Thanks,
Gavin

0 Votes 0 ·

so there are two sep DAGS and each DAG has only 2 mailbox servers as members?

0 Votes 0 ·

Correct, the part that makes no sense is I tried to move a mailbox from each DAG and had the same problem with both. I have done many moves in the past and not had any issues, the only thing I can think of that has changed recently is that we setup a hybrid setup with O365, but all mailboxes are still on-prem.

Thanks,
Gavin

0 Votes 0 ·

So this is what I did to get the mailbox to migrate. I moved the Migration Mailbox to a different database, restarted the migration from the Exchange cli and the mailbox moved successfully. I am still having issues with the batch migration, but at least I was able to move the mailbox to a new database.

0 Votes 0 ·
LucasLiu-MSFT avatar image
0 Votes"
LucasLiu-MSFT answered ·

Hi @GavinRoss-3568 ,
Did you try to restart the mailbox replication service?

Do you mean that this error occurs only when migrating to a new database? Successful when migrating to an existing database. I create a new database in Exchange hybrid environment, and it succeeded when I moved a mailbox to it. If possible, please try to create a another database and try to migrate again.

If you run the following the first command to move only one mailbox, will there still be the same error? If so, run the second commands to view the detail report of the migration. Especially "ItemsTransferred", "PercentComplete" parameters, check whether the migration is in progress. If you migrate larger or more mailboxes, please wait patiently for some time. And check the specific report to see if there is any relevant information.

 New-MoveRequest -Identity <> -TargetDatabase “DB Name”
 Get-MoveRequest | Get-MoveRequestStatistics -IncludeReport | fl



If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.





· 2 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Seems like I have stumbled onto the main problem, when checking the health of the replication, I am seeing these errors

This was from the move report
3/10/2021 7:53:57 AM [SERVER1] Fatal error MailboxDataReplicationFailedPermanentException has occurred.

And this was when I ran the following;

Get-DatabaseAvailabilityGroup | Select -ExpandProperty:Servers | Test-ReplicationHealth | Where {$_.Result.Value -ne "Passed"}

Server Check Result Error


SERVER1 DatabaseAvailability FAILED Failures:...
SERVER2 DatabaseAvailability FAILED Failures:...

Any ideas on how to get this working again?

0 Votes 0 ·

what are the specific failures?

0 Votes 0 ·
GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 answered ·

This is the error on all the databases in the DAG

SERVER1:
Server 'SERVER1.domainname.com' component (HighAvailability) state is offline. If you need to activate databases copies on this server, you can use Set-ServerComponentState
-Component 'HighAvailability' -State 'Active' and retry Move-ActiveMailboxDatabase.

                     There were database availability check failures for database 'SERVER1-MB02' that may be lowering its availability. Availability Count: 1. Expected Availability Count: 2. Detailed error(s):

When I check the status, it shows as active?

SERVER1.domainname.com HighAvailability Active

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

What do see for
Get-ServerComponentState <server>

0 Votes 0 ·
GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 answered ·

[PS] C:\WINDOWS\system32>Get-ServerComponentState SERVER1

Server Component State


SERVER1.domainname.com ServerWideOffline Active
SERVER1.domainname.com HubTransport Active
SERVER1.domainname.com FrontendTransport Active
SERVER1.domainname.com Monitoring Active
SERVER1.domainname.com RecoveryActionsEnabled Active
SERVER1.domainname.com AutoDiscoverProxy Active
SERVER1.domainname.com ActiveSyncProxy Active
SERVER1.domainname.com EcpProxy Active
SERVER1.domainname.com EwsProxy Active
SERVER1.domainname.com ImapProxy Active
SERVER1.domainname.com OabProxy Active
SERVER1.domainname.com OwaProxy Active
SERVER1.domainname.com PopProxy Active
SERVER1.domainname.com PushNotificationsProxy Active
SERVER1.domainname.com RpsProxy Active
SERVER1.domainname.com RwsProxy Active
SERVER1.domainname.com RpcProxy Active
SERVER1.domainname.com UMCallRouter Active
SERVER1.domainname.com XropProxy Active
SERVER1.domainname.com HttpProxyAvailabilityGroup Active
SERVER1.domainname.com ForwardSyncDaemon Inactive
SERVER1.domainname.com ProvisioningRps Inactive
SERVER1.domainname.com MapiProxy Active
SERVER1.domainname.com EdgeTransport Active
SERVER1.domainname.com HighAvailability Active
SERVER1.domainname.com SharedCache Active
SERVER1.domainname.com MailboxDeliveryProxy Active
SERVER1.domainname.com RoutingUpdates Active
SERVER1.domainname.com RestProxy Active
SERVER1.domainname.com DefaultProxy Active
SERVER1.domainname.com Lsass Active
SERVER1.domainname.com RoutingService Active
SERVER1.domainname.com E4EProxy Active
SERVER1.domainname.com CafeLAMv2 Active
SERVER1.domainname.com LogExportProvider Active

Everything looks fine?

·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

AndyDavid avatar image
0 Votes"
AndyDavid answered ·

Had a similar thread just recently:
https://docs.microsoft.com/en-us/answers/questions/287972/exchange-2016-test-replicationhealth-databaseavail.html

Run:
Get-ServerComponentState -Identity <server> | FL

then see if that thread is related

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Everything shows as active? (see attached)76389-export.txt


0 Votes 0 ·
export.txt (15.0 KiB)
GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 answered ·

So it appears that the problem was related to the other issue you were helping me with regarding the missing Self Signed Cert, after I fixed that issue, the replication errors went away as well.

[PS] C:\WINDOWS\system32>Test-ReplicationHealth -Identity SERVER1

Server Check Result Error


SERVER1 ClusterService Passed
SERVER1 ReplayService Passed
SERVER1 ActiveManager Passed
SERVER1 TasksRpcListener Passed
SERVER1 TcpListener Passed
SERVER1 ServerLocatorService Passed
SERVER1 DagMembersUp Passed
SERVER1 MonitoringService Passed
SERVER1 ClusterNetwork Passed
SERVER1 QuorumGroup Passed
SERVER1 FileShareQuorum Passed
SERVER1 DatabaseRedundancy Passed
SERVER1 DatabaseAvailability Passed
SERVER1 DBCopySuspended Passed
SERVER1 DBCopyFailed Passed
SERVER1 DBInitializing Passed
SERVER1 DBDisconnected Passed
SERVER1 DBLogCopyKeepingUp Passed
SERVER1 DBLogReplayKeepingUp Passed

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

<thumbs up> 'Glad to see things coming together!

0 Votes 0 ·
GavinRoss-3568 avatar image
0 Votes"
GavinRoss-3568 answered ·

The migration of the mailbox completed as well. Seems like it was all related to the missing certificate on the server.

· 1 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @GavinRoss-3568 ,
I am glad to hear that your issue has been resolved.

If possible, you could click “Accept as answer” in your free time to mark helpful reply as an answer, this will make answer searching in the forum easier and be beneficial to other community members as well.



If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


0 Votes 0 ·