You can run the below cmdlet to return and delete public folder mailboxes for the Secondary Hierarchy type:
Get-Mailbox -PublicFolder | where {$_.IsRootPublicFolderMailbox -eq $False} | Remove-Mailbox -PublicFolder
And for Primary Hierarchy type, you can use:
Get-Mailbox -PublicFolder | where {$_.IsRootPublicFolderMailbox -eq $true} | Remove-Mailbox -PublicFolder
Then you can go into ADSI and edit.
Browse to Configuration\Services \Microsoft Exchange\ {org name} and view the properties of the org.
Now Find msExchDefaultPublicFolderMailbox and clear it.
If the issue persists, you could shut down the server host PFs and mark sure it will not affect Exchange Online PFs. Then remove the Exchange server from ADSI.
I hope this will help you.