Azure AD

How to Get Microsoft 365 Policy Operations Report in Azure AD

How secure is your Entra ID environment during policy changes? Even minor adjustments in your organization’s policies can accidentally lead to unauthorized access, exposing sensitive data. By keeping track of all policy operations, you can quickly spot these changes, investigate the cause, and take action. This guide covers the most effective methods to monitor and manage policy configurations in Microsoft 365 to keep your organization secure and compliant.

Using Microsoft Purview Portal

Microsoft 365 Permission Required

Authentication Administrator Least Privilege

Global Admin Most Privilege

Log in to the Microsoft 365 Purview portal.
Navigate to the Audit log search tab under Solutions.
Now, customize the date range if required. Then, enter the following operations in the "Activities - operation names" field as comma-separated values.

Add Default Policy Application,Add Policy Service Principal,Add Policy,Add Policy to Service Principal,Delete Policy,Remove Default Policy Application,Remove Default Policy Service Principal,Remove Policy Credentials,Remove Policy from Service Principal,Update Policy
Now, click on "Search". Once it is completed, open the result and use the 'Export' button to download the Azure AD policy operations report.

Here, you can monitor changes made to the following M365 policy operations:

Sign-in Risk Policy
User Risk Policy
MFA Registration Policy
Conditional Access Policy
B2B Collaboration Policy

Using Windows PowerShell

Microsoft 365 Permission Required

Authentication Administrator Least Privilege

Global Admin Most Privilege

Connect to Exchange Online PowerShell using the below cmdlet.
Windows PowerShell
```
 Connect-ExchangeOnline
```
Run the below PowerShell cmdlet to get all Microsoft 365 policy operations.

Windows PowerShell

 Search-UnifiedAuditLog -StartDate "MM/DD/YYYY" -EndDate "MM/DD/YYYY" -Operation "Policy" | Format-Table CreationDate, Operations, UserIds, RecordType -AutoSize

PowerShell cmdlets provide limited information for tracking policy related changes in Microsoft 365.
Although they show the actions performed, they don't display the policy names involved. This limitation makes it challenging for admins to track the changes made to specific policies.

Using AdminDroid
Simple & Quick

AdminDroid Permission Required Delegated

Any user with report access delegated by the Super Admin.

Log in to the AdminDroid Office 365 reporter.
Navigate to the All Policy Operations report under Audit»Azure AD»Policy Audit to review all Microsoft 365 policy operations.

AdminDroid Azure AD Auditing Tool

Ensure maximum control by monitoring comprehensive Azure AD policy changes!

Introducing AdminDroid’s Azure AD management tool, designed to enhance your Microsoft 365 policy reporting and auditing processes. With just a few clicks, you can customize advanced reports by adding columns from other reports. This tool streamlines your administrative tasks and offers greater efficiency than native methods.

Monitor Policy Deletions in M365 for Enhanced Security

Keep track of policy deletions in Microsoft 365 to ensure important policies are not removed. This practice enhances security and improves administrative oversight.

Keep an Eye on Default Policy Assignments

Monitor Microsoft 365 policy assignment operations and ensure they are correctly assigned to the appropriate applications or service principals.

Enhanced Security Through Conditional Access Policies

Monitor Conditional Access policies with grant controls, to ensure that access conditions, such as requiring compliant devices or secure sign-in locations, are effectively enforced.

Graphical Charts for Comprehensive M365 Policy Monitoring

Use interactive graphical charts to monitor policy operations in real-time. Visualize compliance levels to quickly identify trends and deviations. This approach enables prompt corrective actions when needed.

Track All Confirmed Risky Sign-ins in Microsoft 365

Stay on top of security by monitoring all confirmed risky sign-ins. This lists sign-ins that have been remediated, dismissed, or flagged as compromised. You can then respond quickly to potential threats.

Export Detailed MFA Assignments for Conditional Access Policies

Get valuable insights into the included and excluded conditions for MFA-enabled Conditional Access policies, and export them in various formats, such as CSV or PDF.

In summary, AdminDroid enhances Azure AD policy management by:

Resolving M365 Policy Misconfigurations: Quickly addresses real-time issues related to policy misconfigurations that could lead to unauthorized access.
Streamlining Audit Processes: Automates the collection and analysis of audit logs related to Microsoft Entra ID policies, reducing manual effort and enabling faster identification of compliance gaps.
Boosting Efficiency: Provides comprehensive auditing and alerting features for policy-related activities. This enables you to take action and protect your data.

Explore a full range of reporting options

Download Live Demo

Important Tips

Monitor Conditional Access policy changes to ensure security by identifying and investigating suspicious modifications, such as someone weakening a policy to grant unauthorized access.

Turn on Microsoft 365 auto-claim policies to automate license assignments when users first access an application, reducing manual admin tasks and saving time.

Secure your Microsoft 365 data with device-based Conditional Access policies. Enforce security protocols on devices to ensure compliance and prevent unauthorized access.

Common Errors and Resolution Steps

The following are the possible errors and troubleshooting hints while dealing with Azure AD policy operations.

Error You cannot access this right now.

This error occurs when you try to access a resource owned by another organization, which has implemented risk-based policies to block risky users.

Fix Contact your admin to get your risk level reviewed and remediate it to allow access to other organization’s resources.

Error Failure to update policy due to object limit.

This error will occur when configuring cross-tenant access settings and you have reached the policy object limit of 25 KB.

Fix To address this issue, review the current cross-tenant access policy to assess the number of objects included. Consider optimizing object usage by removing or consolidating unnecessary objects, breaking down policies into smaller parts, and monitoring the policy size to avoid reaching the limit.

Error AADSTS530032 - Access has been blocked by Conditional Access policies. The access policy does not allow token issuance.

This error occurs when Conditional Access policies are configured to block access based on specific conditions, such as location, device, or user risk, preventing token issuance.

Fix Review the Conditional Access policies configured in your Microsoft Entra tenant. Ensure that the policies are not overly restrictive and are correctly set up to allow access under the conditions you are attempting. Adjust the policy settings if necessary or consult with your Azure AD administrator to modify the policy to permit the required access.

Error New-AzureADPolicy : Error occurred while executing NewPolicy.

This error can occur due to syntax errors in PowerShell cmdlets or insufficient permissions to create policies in Azure AD.

Fix New-AzureADPolicy cmdlet requires the necessary permissions to create policies in Azure AD. Ensure your account has the right permissions, as lacking them will cause the cmdlet to fail.

Entra ID

Frequently Asked Questions

Bolster Your Microsoft 365 Security by Monitoring All Azure AD Policy-Related Operations

What types of policy operations can be monitored in Azure AD?

How to create and manage external identities in Microsoft Entra?

How to monitor cross-tenant access policy changes in Azure AD?

How to manage Microsoft Entra ID protection policies?

How to check which conditional access policy is blocking user sign-ins?

1. What types of policy operations can be monitored in Azure AD?

In Entra ID, it is crucial to monitor key policy types to manage access and ensure security, especially in high-risk environments. For instance, unnoticed policy changes could lead to a data breach. To prevent such incidents, it's essential to consistently track and manage these changes.

The primary policies to focus on include:

Identity Protection Policies
Access Review Policies
Conditional Access Policies
B2B Collaboration Policies

To ensure these policies are managed effectively, it is important to monitor the following operations:

Creation of New Policies: Track when new policies are created, including critical ones like Conditional Access policies. Regularly auditing and exporting CA policies allows you to detect unauthorized changes. This helps safeguard your environment against potential security risks and ensures compliance with organizational guidelines.
Modification of Existing Policies: Monitor policy updates to identify potential misconfigurations, such as accidentally disabling multi-factor authentication to ensure that policies adapt to evolving business needs without compromising security & compliance.
Deletion of Policies: Keep an eye on deleted policies to quickly identify gaps in enforcement and ensure that critical operational controls are not unintentionally removed.
Assignments and Removals: Track the assignment of policies to users or groups and any subsequent removals to maintain appropriate access control.

How to create alerts for policy operations in Azure AD? AdminDroid's alerting feature enables you to effortlessly set up alerts for your Entra ID policies with just one click.

Click the alert icon once and experience the magic of real-time event notifications!

2. How to create and manage external identities in Microsoft Entra?

External identities in Microsoft Entra is a critical component in protecting user identities with risk-based policies. It focuses on detecting and responding to potential threats through user risk and sign-in risk policies. Additional policies, such as Conditional Access and MFA registration, further enhance overall security.

Cross-Tenant Access Settings:

Cross-tenant access settings in B2B (Business-to-Business) collaboration enable organizations to manage how users from external organizations access resources in your tenant. This includes configuring inbound and outbound access settings to define what external users can access in your environment, providing secure collaboration between tenants.

Here’s how it works:

Inbound access: This controls how you allow users (guests) from external Microsoft Entra tenants to access resources in your home tenant. You can choose to apply these settings broadly or to specific users, groups, or applications.
Outbound access: This governs how your internal users can access resources in external Microsoft Entra tenants as guests. Similar to inbound access, you can apply these settings to all users or target specific individuals, groups, or applications.

External Collaboration Settings:

Microsoft Entra external collaboration settings are essential for managing guest user access securely. These settings allow you to control guest invitations and restrict external users.

Guest User Restrictions: To manage who can invite external users and what they can access, you can implement guest user restrictions. This limits the ability of unauthorized users to invite guests and ensures that external users are properly reviewed. This can be customized to suit your organization’s needs by setting rules that restrict guest user invitations to certain users or groups.
Tenant Allow/Block List Policy: Manage external collaboration by controlling external domains which are to be allowed or blocked from interacting with your organization. Set tenant allow/block list policies to whitelist or blacklist specific domains. This ensures that only trusted external users from approved domains can access your resources.

Access Reviews:

Set up access review policies to periodically assess user permissions and ensure appropriate access levels. By regularly reviewing access, you can ensure that inactive guest accounts, which could be potential security risks, are removed. This process not only enhances security but also helps optimize license usage by eliminating unnecessary accounts.

3. How to monitor cross-tenant access policy changes in Azure AD?

Monitoring B2B collaboration cross-tenant access policy changes in Azure AD is crucial for maintaining secure interactions with external partners, suppliers, and clients. By keeping track of these policy changes, you can ensure that external users have appropriate access levels while preventing unauthorized access.

To Monitor B2B Collaboration Policy Changes in Azure AD:

Sign in to the Microsoft Entra admin center.
In the left-hand navigation pane, click on Monitoring & Health. Under this section, select Audit logs.
Use the Category filter to refine the logs related to Cross-Tenant Access Settings.
Here, you can view changes made to cross-tenant access settings, such as policy modifications or new organization configurations.
You can also use 'Date Range' and 'Initiated by' filters to refine your search further.

4. How to manage Microsoft Entra ID protection policies?

Microsoft Entra ID protection is critical for safeguarding user identities by applying risk-based policies that detect and respond to potential threats. It allows you to configure policies for both sign-in risk and user risk. These policies help mitigate unauthorized access by enforcing multi-factor authentication (MFA) or requiring password resets.

Below are the key policies that contribute to strengthening your organization’s security posture:

Risk Policies:

Configure user and sign-in risk policies to remediate risks by identifying compromised user accounts and risky sign-ins.

User Risk Policies: These policies detect unusual behaviors, such as unfamiliar devices or locations, triggering automatic actions like password resets or access blocks.
Sign-in Risk Policies: These policies evaluate the risk of individual logins, requiring additional verification (e.g., MFA) for sign-ins from unfamiliar IP addresses or suspicious locations to ensure only legitimate users access resources.
Risk-based CA Policies: Deploy risk-based Conditional Access policies to effectively manage risky sign-ins. Standard risk policies typically include measures like multi-factor authentication (MFA) and password resets. However, advanced risk-based CA policies provide more detailed control and additional options, such as adaptive access rules and custom risk levels, to address complex scenarios with greater precision.

MFA Registration Policy:

The MFA registration policy ensures that all users are enrolled in multi-factor authentication, adding an extra layer of security to protect against credential-based attacks. If a user is logging in from a unknown device or location, the MFA registration ensures that they have an additional verification method (such as a text message or authentication app) to confirm their identity.

How to Configure Multi-factor Authentication Registration Policy

Log in to the Microsoft Entra admin center.
Navigate to the Protection»Identity Protection» Multifactor authentication registration policy.
In the Assignments section, click on 'All Users'.
- Under Include, choose either 'All users' or 'Select 'Individuals and groups' to target specific users.
- Under Exclude, designate 'Users and groups' that should be excluded, such as your emergency access or break-glass accounts.
Ensure Policy enforcement is set to Enabled. Then, click 'Save'.

5. How to check which conditional access policy is blocking user sign-ins?

Updating Conditional Access policies can sometimes lead to unexpected sign-in issues for Microsoft 365 users. For instance, a new location-based restriction might inadvertently block legitimate access. By analyzing Conditional Access policies and their sign-in impacts you can identify which specific policy is blocking user sign-ins.

To identify which Conditional Access policy is blocking user sign-ins:

Sign in to the Microsoft Entra admin center.
Navigate to Identity»Monitoring & health»Sign-in logs.
Apply the following filter to find the sign-ins blocked by conditional access policies:
- Status: Choose “Failure” to focus on failed attempts.
- Conditional Access: Apply this filter and select “Failure” to see which policies were applied during the failed sign-in attempts.
Once done, select the specific failed sign-in event and go to the Conditional Access tab to view those data.

Bonus tip: You can also view Conditional Access sign-in logs in Entra ID by navigating to Identity»Protection»Conditional Access»Sign-in logs.

AdminDroid’s User Sign-ins Blocked by Conditional Access Policy report offers an easy way to monitor all sign-ins blocked by Conditional Access policies across your organization.

Note📝: The ‘Blocking CA Policy Name’ column in this report shows the specific Conditional Access policy responsible for blocking user sign-ins.

Additionally, if you need to share the report with someone, you can effortlessly email it with just a few clicks using the convenient 'Email this report now' option.