since disclosure of the on-prem Exchange Server vulnerabilities we are facing a huge wave of global sign-in attempts to 'Office 365 Exchange Online'. This put our user accounts at serious risk and causes permanent user lock-outs. We want to mitigate this thread by using Azure AD Conditional Access policies to protect our users and prevent sign-ins from specific countries, which we haven't used so far.
We have created the policies with the help of https://docs.microsoft.com/en-us/azure/active-directory/conditional-access/location-condition but in any reason, they don't apply. The rules are pretty straightforward:
User and Groups: Selected users for which the policy must apply
Cloud apps or actions: All cloud apps
Conditions: Locations --> Include --> Selected locations: A set of countries, for which we want to block access
Grant: Block access
The policy is 'report-only' for the moment. When I test the application/trigger of the policy using the What-If testing tool, the policy is supposed to apply. We are monitoring the sign-ins for the selected users since then, but the policy is not applied/triggered in the Conditional access column. As a sidenote, we also created a second policy to enforce multi-factor-authentication but it doesn't apply either.
To rule out that this is a licensing issue: We use an Azure AD Premium P1 license and an Office E1 + EMS E3 on the user site.
We would be glad if you can offer any support on this issue! If you need further information to investigate, please let me know.