Email destined for recently moved Modern Public Folders are going to the Poison queue.
I recently worked a case where the customer had moved all of the public folders from one Modern Public Folder mailbox to another and deleted the old Public Folder mailbox. This was on Exchange 2013 CU9. After he did that, email destined for what were previously mail-enabled Public Folders started to show up in the Poison queue. When we did a get-mailpublicfolder against one of the Public Folders (lets call it pf using SMTP address firstname.lastname@example.org) we found the following:
ContentMailbox : instant-grits.com/Deleted Objects/PFMbx01
The correct email address is specified under the EmailAddresses property.
So here's the problem; Exchange still sees the mailbox content as being on the Deleted Object, aka the old Public Folder Mailbox server.
We also noted that the new public folder was not automatically being mail-enabled after being migrated from one Public Folder Mailbox to the other. So we tried to mail-enable it using the enable-mailPublicFolder. It mail-enabled, but the email addresses it assigned were enumerated, so not the original email address. This will cause mail stuck in the poison queue to never be able to successfully deliver to the new folder.
Why is this happening to me?
In a nutshell, the Microsoft Exchange System Object (MESO) that exists for that public folder is still attached to the deleted public folder. Since the MESO holds the mail properties for a mail-enabled object, mail-enabling the new folder will not be able to use the old address. Since email@example.com is not available, the new object gets an enumerated version of it, so firstname.lastname@example.org. Since that does not match the address, and there is no X500 pointed to the deleted folder, Exchange does not know how to deliver it. Since the old address does exist, Exchange will try, but it will then go to the Poison queue as the Folder it is trying to go to is a deleted object.
So what do I do about it?
As always, you ask the best questions. Let's assume the public folder is in the following location in the public folder tree: \thingsthatgowithgrits\shrimp\pf.
There are 5 steps to cleaning this up and getting your messages in the Poison queue and new messages to deliver successfully to the Public Folders in the new Public Folder Mailbox:
- Mail-disable all of the public folders in the new Public Folder Mailbox. You can do this from the Exchange Admin Center (EAC), or by using the disable-mailpublicfolder cmdlet using the full public folder path of the public folder, like such:disable-mailpublicfolder "\thingsthatgowithgrits\shrimp\pf"
- Once all of the public folders in the new Public Folder Mailbox have been mail-disabled, you need to find and delete all of the MESO objects that have the same name as the public folders you are cleaning up. Make sure you capture the SMTP address of those MESO objects. They should be in the EmailAddresses property of the get-mailpublicfolder cmdlet you ran earlier. A quick script that Bill Long wrote and I modified very slightly can dump all of the MESO objects to a Text file. From there you can sort it and go to Active Directory Users and Computer. Make sure you have it in Advanced view. Then browse to the Microsoft Exchange System Objects container, which is at the root of the domain, and delete all of the appropriate MESO objects. The script looks as follows: $mesoDN = "CN=Microsoft Exchange System Objects,DC=instant-grits,DC=com"
$mesoContainer = [ADSI]("LDAP://" + $mesoDN)
$sysMbxFinder = new-object System.DirectoryServices.DirectorySearcher
$sysMbxFinder.SearchRoot = $mesoContainer
$sysMbxFinder.PageSize = 1000
$sysMbxFinder.Filter = "(cn=*)"
$sysMbxFinder.SearchScope = "OneLevel"
$sysMbxResults = $sysMbxFinder.FindAll()
$sysMbxResults.Path > c:\MESOObjects.txt
- Once all of the appropriate MESO objects have been deleted, wait until Active Directory has finished replicating the change. For single site environments, this is pretty instant. For multi-site environments, it will depend on your AD topology. It will take as long as your longest and slowest route. Ask your network guy how long that should be.
- Now that we are fully replicated, we can mail-enable the public folders. You can do this from the Exchange Admin Center (EAC), or by using the enable-mailpublicfolder cmdlet using the full public folder path of the public folder, like such:enable-mailpublicfolder "\thingsthatgowithgrits\shrimp\pf" Using the enable-mailpublicfolder cmdlet in my lab has produced mixed results, sometimes it works, sometimes it doesn't, likely due to AD replication or my Exchange environment being on too old of a Cumulative Update (CU). So be prepared to do it manually from the EAC.
- Once all of the public folders that are going to be mail-enabled have been, you can then go into the queue viewer and browse the poison queue. From there you should be able to select all messages and then click on Resume. If the resume fails for any reason, you either missed cleaning up one or more of the public folders, or there are messages that are in the poison queue for some other reason. At that time you are ready to bounce the transport service. You can do that from the Exchange Management Shell with the following command:
After restarting the queue, you can wait for the Poison Queue to reattempt deliver on its messages. All that have properly cleaned up public folders should then deliver.
I do strongly urge you to get and keep your Exchange servers at a fully supported cumulative update level. That level is N-1. So the latest version minus 1. If we are currently at CU16, then you need to be at least on CU15.
Thanks to Bill Long for his impressive blogs on the topics of MESO objects. The one in particular that I referenced in this blog is as follows:
Also thanks Matthew Huynh for offering up his lab to me so I can test the scenario.
I have tested this scenario in CU15 and I could not reproduce it. So before you move any public folders across public folder mailboxes, it is highly recommended that you get to the latest Cumulative Update first.