Azure AD Conditional Access policies are not applied

Question

Hi there,

since disclosure of the on-prem Exchange Server vulnerabilities we are facing a huge wave of global sign-in attempts to 'Office 365 Exchange Online'. This put our user accounts at serious risk and causes permanent user lock-outs. We want to mitigate this thread by using Azure AD Conditional Access policies to protect our users and prevent sign-ins from specific countries, which we haven't used so far.

We have created the policies with the help of https://learn.microsoft.com/en-us/azure/active-directory/conditional-access/location-condition but in any reason, they don't apply. The rules are pretty straightforward:

• User and Groups: Selected users for which the policy must apply
• Cloud apps or actions: All cloud apps
• Conditions: Locations --> Include --> Selected locations: A set of countries, for which we want to block access
• Grant: Block access
• Session: -

The policy is 'report-only' for the moment. When I test the application/trigger of the policy using the What-If testing tool, the policy is supposed to apply. We are monitoring the sign-ins for the selected users since then, but the policy is not applied/triggered in the Conditional access column. As a sidenote, we also created a second policy to enforce multi-factor-authentication but it doesn't apply either.

To rule out that this is a licensing issue: We use an Azure AD Premium P1 license and an Office E1 + EMS E3 on the user site.

We would be glad if you can offer any support on this issue! If you need further information to investigate, please let me know.

Kind regards,
Fabian Schlegel

Answer 1

From what I can tell, the Conditional Access policies are working as expected. This is because they are only applied to successful sign-ins. If a sign-in is successful, then the policies are evaluated to determine if the user should be able to access those resources.

If you want to keep the sign-in attempt from getting that far, your best bet is to determine the type of connection being made to Exchange (SMTP, IMAP, POP, etc) and disable it. Exchange Online does a pre-auth step before sending the sign-in to Azure, so if you disable those methods in Exchange Online, it won't forward the request to Azure. Just make sure you don't disable something you need like SMTP for emailing scans. Or you can go back and enable it for those few accounts that need it while leaving it restricted for all others.