@Kostas Backas ,For our issue, please confirm if any other policies which add user to admin group are deployed to the device. Meanwhile, we can test on some more windows 11 devices to see if it have the same issue.
To remove the user from the device, we can use LocalUsersAndGroups CSP to remove the member. Here is a link for the reference:
https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-localusersandgroups
Hope it can help.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.