🎉 Our Microsoft 365 Reporting & Management Tool is available in Marketplace 🚀
Microsoft Entra ID

How to Analyze Conditional Access Policy Sign-in Impacts with Entra ID

Planning and implementing Conditional Access policies in your organization is very challenging. Overly restrictive policies may frustrate users, while lenient policies can create security vulnerabilities. For example, setting strict sign-in frequencies for employees who work remotely can hinder productivity due to repeated access denials. This is why analyzing Conditional Access policies and their sign-in impacts is crucial. Learn to balance security and usability to ensure robust protection without compromising user experience.

Microsoft 365 tools
Solution 1
Solution 2
Solution 3
Solution 4
Important Tips
Errors and Resolution Steps

Using Conditional Access Insights and Reporting Workbook

Microsoft 365 Permission Required
Security Reader Least Privilege
Global Admin Most Privilege
  • Sign in to the Microsoft Entra admin center and navigate to Identity»Monitoring & health»Workbooks.
  • Under the Conditional Access section, select the Conditional Access Insights and Reporting Workbook.
  • In the workbook, you'll find two sections: User sign-ins and Service Principal sign-ins.
  • To view both data under these sections, ensure that diagnostic settings are configured to send SignInLogs and ServicePrincipalSignInLogs to your Log Analytics workspace.
  • These reports allow admins to track sign-in activity, identify trends, and pinpoint anomalies in user and service principal behaviors, helping to detect potential security threats or policy misconfigurations.
  • This workbook aggregates sign-in results to help admins understand the impact of Conditional Access policies, highlighting users who bypass them and enabling deeper analysis of security posture and policy effectiveness.
Using Conditional Access Insights and Reporting Workbook

Using Sign-in Logs in Entra Admin Center

Microsoft 365 Permission Required
Security Reader Least Privilege
Global Admin Most Privilege
  • Log in to the Microsoft Entra admin center.
  • Navigate to the Identity»Monitoring & health»Sign-in logs.
  • Select the sign-in event you wish to monitor.
  • Go to the Conditional Access tab to see which CA policies were applied to the selected sign-in event and analyze their impacts.
Using Sign-in Logs in Entra Admin Center

Using Windows PowerShell

Microsoft 365 Permission Required
Security Reader Least Privilege
Global Admin Most Privilege
  • Connect to the Microsoft Graph PowerShell module using the below cmdlet.
  • Windows PowerShell Windows PowerShell 
     Connect-MgGraph
  • Run the PowerShell command below to retrieve users' sign-in logs impacted by Conditional Access Policies. Provide the target file path for the -Path parameter to export the report and review the Conditional Access Policy impacts.
  • Windows PowerShell Windows PowerShell 
     Get-MgAuditLogSignIn | ForEach-Object {  $AppliedPolicies = ($_.AppliedConditionalAccessPolicies | Where-Object { $_.Result -in @('Success', 'Failed') } | ForEach-Object { $_.DisplayName }) -join ","   
if (-not $AppliedPolicies) { $AppliedPolicies = "None" }
    
    [PSCustomObject]@{
        CreatedDateTime = $_.CreatedDateTime
        UserPrincipalName = $_.UserPrincipalName
        AppDisplayName = $_.AppDisplayName
        AppliedPolicies = $AppliedPolicies
        StatusCode = $_.Status.ErrorCode
        StatusFailureReason = $_.Status.FailureReason
        StatusAdditionalDetails = $_.Status.AdditionalDetails
        City = $_.Location.City
        State = $_.Location.State
        CountryOrRegion = $_.Location.CountryOrRegion
     GeoCoordinates = $_.Location.GeoCoordinates
   AppId = $_.AppId
    } | Export-Csv -Path '<OutputFilePath>' -NoTypeInformation -Append -Force
}
Using Windows PowerShell

Effectively Audit Conditional Access Impacts on Microsoft 365 User Sign-ins

AdminDroid's Azure AD Analytics tool offers a comprehensive analysis of Conditional Access, covering not just sign-ins but also policy assignments, conditions, MFA details, and more. With AdminDroid, monitoring Conditional Access sign-ins becomes straightforward, helping you prevent policy misconfigurations that could compromise your organization's security.

Track CA Blocked Sign-ins with AdminDroid Alerts

Get instant notifications from AdminDroid's default alert policies when an unusual number of sign-ins are blocked by Conditional Access, ensuring quick resolution during critical times.

View Conditional Access Sign-in Counts with AdminDroid

Monitor the summary of sign-in counts by CA policies applied to detect unusual login patterns, such as spikes in failed sign-ins from specific locations or devices.

Discover Excluded Users from Conditional Access Policies with AdminDroid

By monitoring Conditional Access policies with the user conditions report, admins can identify excluded users to ensure only essential accounts are excluded, and all other user accounts are properly included.

Automate CA Reports to Review Sign-ins Failed by Report-only CA Policies

Use the AdminDroid’s advanced scheduling feature to receive regular reports on sign-ins failed by report-only Conditional Access policies, ensuring users comply with security protocols and access resources appropriately.

Verify MFA Configuration within Conditional Access Policies

List Conditional Access policies with MFA grant control to ensure applicable users have enabled MFA, thereby securing access, preventing unauthorized logins, and maintaining compliance.

Monitor Guest User CA Failures with AdminDroid Reports

View a summary of failed Conditional Access sign-ins by user type to effectively monitor guest user access, identify potential risks, and enforce stricter controls in your Microsoft 365 environment.

Monitoring Azure AD Conditional Access sign-ins with AdminDroid boosts your security by providing detailed insights into policy enforcement and potential issues. Stay proactive and informed, making swift adjustments to optimize your CA policies in Microsoft 365. AdminDroid simplifies tracking, saving time, and ensuring robust protection.

Important Tips

Use the Impact Analysis of Risk-Based Access Policies workbook to monitor the effects of risk-based Conditional Access policies during each sign-in in Entra ID.

Exclude your break glass accounts from Conditional Access policies to recover during account lockouts, but carefully evaluate these exclusions to avoid accidental blocks.

Regularly monitor Conditional Access policy changes to understand their impact on sign-in behavior and quickly identify any issues or unintended consequences.

Common Errors and Resolution Steps

The following are possible errors and troubleshooting hints while tracking Conditional Access sign-in logs.

Error You cannot access this right now. Your sign-in was successful but does not meet the criteria to access this resource. For example, you might be signing in from a browser, app, or location that is restricted by your admin. Error Code: 53003

This error occurs when a sign-in attempt is blocked due to Conditional Access restrictions.

Fix Check your configured Conditional Access policies, especially the ones with blocked conditions, update your Microsoft apps, and ensure that the sign-in location is supported by your organization.

Error This device does not meet your organization’s compliance requirements. Open your organization’s device management portal to take action. Error Code: 53000

This error occurs when a Conditional Access policy requires a compliant device, but the device you used to sign in isn't compliant.

Fix Enroll your device with an approved MDM provider, such as Intune. Then, try to sign in again.

Error Device is not in the required state. Error Code: 53001

This error occurs when a Conditional Access policy requires a domain-joined device, but the device is not domain-joined.

Fix Ensure the user signs into your organization with a domain-joined device.

Error Application used is not an approved application for conditional access. Error Code: 53002

This error occurs when the user tries to use an app that isn't approved for Conditional Access.

Fix Ensure the user accesses one of the approved applications from the provided list to gain access.

Error Sign-in error code 50058.

This error occurs when the user attempts to sign in but fails to complete the sign-in process.

Fix Ensure the user completes the sign-in process.

Error Sign-in error code 90025.

This error occurs when an internal Microsoft Entra service exceeds its retry allowance to sign the user in.

Fix This error often resolves itself without the user noticing. If the issue persists, ask the user to sign in again.

Error Sign-in error code 500121.

This error occurs when the user doesn't complete the MFA prompt.

Fix Instruct the user to complete the MFA setup process to sign in.

Kickstart Your Journey With
AdminDroid

Your Microsoft 365 Companion with Enormous Reporting Capabilities

Download Now
Browse All Guides
User Help Manuals Compliance Docs Customer Stories
All Guides 99+
Microsoft Entra ID
Analyze Conditional Access Sign-ins Report in M365
Microsoft Entra ID Guides