Hello MohdArif,
I think your issue resides that you want to use a redundancy of AGPM servers. The architecture of AGPM is intended for a unique point and centralized management of policies, thus sharing permissions between servers is not expected. You can indeed create different AGPM servers with different policy scopes, sites, domains, etc.. but they should only manage their own set.
I can recommend the next guide on how to migrate or move the management and archive from/to servers to verify your steps: https://learn.microsoft.com/en-us/microsoft-desktop-optimization-pack/agpm/move-the-agpm-server-and-the-archive
Also this general checklists article for different operations on AGPM: https://learn.microsoft.com/en-us/microsoft-desktop-optimization-pack/agpm/checklist-administer-the-agpm-server-and-archive-agpm40
--------------------------------------------------------------------------------------------------------------------------------
--If the reply is helpful, please Upvote and Accept as answer--