Identify advanced threats with User and Entity Behavior Analytics (UEBA) in Azure Sentinel
- The UEBA and Entity Pages features are now in General Availability in all Azure Sentinel geographies and regions.
What is User and Entity Behavior Analytics (UEBA)?
Identifying threats inside your organization and their potential impact - whether a compromised entity or a malicious insider - has always been a time-consuming and labor-intensive process. Sifting through alerts, connecting the dots, and active hunting all add up to massive amounts of time and effort expended with minimal returns, and the possibility of sophisticated threats simply evading discovery. Particularly elusive threats like zero-day, targeted, and advanced persistent threats can be the most dangerous to your organization, making their detection all the more critical.
The UEBA capability in Azure Sentinel eliminates the drudgery from your analysts’ workloads and the uncertainty from their efforts, and delivers high-fidelity, actionable intelligence, so they can focus on investigation and remediation.
As Azure Sentinel collects logs and alerts from all of its connected data sources, it analyzes them and builds baseline behavioral profiles of your organization’s entities (users, hosts, IP addresses, applications etc.) across time and peer group horizon. Using a variety of techniques and machine learning capabilities, Sentinel can then identify anomalous activity and help you determine if an asset has been compromised. Not only that, but it can also figure out the relative sensitivity of particular assets, identify peer groups of assets, and evaluate the potential impact of any given compromised asset (its “blast radius”). Armed with this information, you can effectively prioritize your investigation and incident handling.
Inspired by Gartner’s paradigm for UEBA solutions, Azure Sentinel provides an "outside-in" approach, based on three frames of reference:
Use cases: By prioritizing for relevant attack vectors and scenarios based on security research aligned with the MITRE ATT&CK framework of tactics, techniques, and sub-techniques that puts various entities as victims, perpetrators, or pivot points in the kill chain; Azure Sentinel focuses specifically on the most valuable logs each data source can provide.
Data Sources: While first and foremost supporting Azure data sources, Azure Sentinel thoughtfully selects third-party data sources to provide data that matches our threat scenarios.
Analytics: Using various machine learning (ML) algorithms, Azure Sentinel identifies anomalous activities and presents evidence clearly and concisely in the form of contextual enrichments, some examples of which appear below.
Azure Sentinel presents artifacts that help your security analysts get a clear understanding of anomalous activities in context, and in comparison with the user's baseline profile. Actions performed by a user (or a host, or an address) are evaluated contextually, where a "true" outcome indicates an identified anomaly:
across geographical locations, devices, and environments.
across time and frequency horizons (compared to user's own history).
as compared to peers' behavior.
as compared to organization's behavior.
Each activity is scored with “Investigation Priority Score” – which determine the probability of a specific user performing a specific activity, based on behavioral learning of the user and their peers. Activities identified as the most abnormal receive the highest scores (on a scale of 0-10).
See how behavior analytics is used in Microsoft Cloud App Security for an example of how this works.
Entities in Azure Sentinel
When alerts are sent to Azure Sentinel, they include data elements that Azure Sentinel identifies and classifies as entities, such as user accounts, hosts, IP addresses and others. On occasion, this identification can be a challenge, if the alert does not contain sufficient information about the entity.
For example, user accounts can be identified in more than one way: using an Azure AD account’s numeric identifier (GUID), or its User Principal Name (UPN) value, or alternatively, using a combination of its username and its NT domain name. Different data sources can identify the same user in different ways. Therefore, whenever possible, Azure Sentinel merges those identifiers into a single entity, so that it can be properly identified.
It can happen, though, that one of your resource providers creates an alert in which an entity is not sufficiently identified - for example, a user name without the domain name context. In such a case, the user entity cannot be merged with other instances of the same user account, which would be identified as a separate entity, and those two entities would remain separate instead of unified.
In order to minimize the risk of this happening, you should verify that all of your alert providers properly identify the entities in the alerts they produce. Additionally, synchronizing user account entities with Azure Active Directory may create a unifying directory, which will be able to merge user account entities.
The following types of entities are currently identified in Azure Sentinel:
- User account (Account)
- IP address (IP)
- Cloud application (CloudApplication)
- Domain name (DNS)
- Azure resource
- File (FileHash)
- Registry key
- Registry value
- Security group
- IoT device
- Mail cluster
- Mail message
- Submission mail
When you encounter any entity (currently limited to users and hosts) in a search, an alert, or an investigation, you can select the entity and be taken to an entity page, a datasheet full of useful information about that entity. The types of information you will find on this page include basic facts about the entity, a timeline of notable events related to this entity and insights about the entity's behavior.
Entity pages consist of three parts:
The left-side panel contains the entity's identifying information, collected from data sources like Azure Active Directory, Azure Monitor, Azure Security Center, and Microsoft Defender.
The center panel shows a graphical and textual timeline of notable events related to the entity, such as alerts, bookmarks, and activities. Activities are aggregations of notable events from Log Analytics. The queries that detect those activities are developed by Microsoft security research teams.
The right-side panel presents behavioral insights on the entity. These insights help to quickly identify anomalies and security threats. The insights are developed by Microsoft security research teams, and are based on anomaly detection models.
The timeline is a major part of the entity page's contribution to behavior analytics in Azure Sentinel. It presents a story about entity-related events, helping you understand the entity's activity within a specific time frame.
You can choose the time range from among several preset options (such as last 24 hours), or set it to any custom-defined time frame. Additionally, you can set filters that limit the information in the timeline to specific types of events or alerts.
The following types of items are included in the timeline:
Alerts - any alerts in which the entity is defined as a mapped entity. Note that if your organization has created custom alerts using analytics rules, you should make sure that the rules' entity mapping is done properly.
Bookmarks - any bookmarks that include the specific entity shown on the page.
Activities - aggregation of notable events relating to the entity.
Entity insights are queries defined by Microsoft security researchers to help your analysts investigate more efficiently and effectively. The insights are presented as part of the entity page, and provide valuable security information on hosts and users, in the form of tabular data and charts. Having the information here means you don't have to detour to Log Analytics. The insights include data regarding Sign-Ins, Group Additions, Anomalous Events and more, and include advanced ML algorithms to detect anomalous behavior. The insights are based on the following data types:
- Audit Logs
- Sign-in Logs
- Office Activity
- BehaviorAnalytics (UEBA)
How to use entity pages
Entity pages are designed to be part of multiple usage scenarios, and can be accessed from incident management, the investigation graph, bookmarks, or directly from the entity search page under Entity behavior analytics in the Azure Sentinel main menu.
Behavior analytics table
|TenantId||unique ID number of the tenant|
|SourceRecordId||unique ID number of the EBA event|
|TimeGenerated||timestamp of the activity's occurrence|
|TimeProcessed||timestamp of the activity's processing by the EBA engine|
|ActivityType||high-level category of the activity|
|ActionType||normalized name of the activity|
|UserName||username of the user that initiated the activity|
|UserPrincipalName||full username of the user that initiated the activity|
|EventSource||data source that provided the original event|
|SourceIPAddress||IP address from which activity was initiated|
|SourceIPLocation||country from which activity was initiated, enriched from IP address|
|SourceDevice||hostname of the device that initiated the activity|
|DestinationIPAddress||IP address of the target of the activity|
|DestinationIPLocation||country of the target of the activity, enriched from IP address|
|DestinationDevice||name of the target device|
|UsersInsights||contextual enrichments of involved users|
|DevicesInsights||contextual enrichments of involved devices|
|ActivityInsights||contextual analysis of activity based on our profiling|
|InvestigationPriority||anomaly score, between 0-10 (0=benign, 10=highly anomalous)|
You can see the full set of contextual enrichments referenced in UsersInsights, DevicesInsights, and ActivityInsights in the UEBA enrichments reference document.
Querying behavior analytics data
Using KQL, we can query the Behavioral Analytics Table.
For example – if we want to find all the cases of a user that failed to sign in to an Azure resource, where it was the user's first attempt to connect from a given country, and connections from that country are uncommon even for the user's peers, we can use the following query:
BehaviorAnalytics | where ActivityType == "FailedLogOn" | where FirstTimeUserConnectedFromCountry == True | where CountryUncommonlyConnectedFromAmongPeers == True
User peers metadata - table and notebook
User peers' metadata provides important context in threat detections, in investigating an incident, and in hunting for a potential threat. Security analysts can observe the normal activities of a user's peers to determine if the user's activities are unusual as compared to those of his or her peers.
Azure Sentinel calculates and ranks a user's peers, based on the user’s Azure AD security group membership, mailing list, et cetera, and stores the peers ranked 1-20 in the UserPeerAnalytics table. The screenshot below shows the schema of the UserPeerAnalytics table, and displays the top eight-ranked peers of the user Kendall Collins. Azure Sentinel uses the term frequency-inverse document frequency (TF-IDF) algorithm to normalize the weighing for calculating the rank: the smaller the group, the higher the weight.
You can use the Jupyter notebook provided in the Azure Sentinel GitHub repository to visualize the user peers metadata. For detailed instructions on how to use the notebook, see the Guided Analysis - User Security Metadata notebook.
Permission analytics - table and notebook
Permission analytics helps determine the potential impact of the compromising of an organizational asset by an attacker. This impact is also known as the asset's "blast radius." Security analysts can use this information to prioritize investigations and incident handling.
Azure Sentinel determines the direct and transitive access rights held by a given user to Azure resources, by evaluating the Azure subscriptions the user can access directly or via groups or service principals. This information, as well as the full list of the user's Azure AD security group membership, is then stored in the UserAccessAnalytics table. The screenshot below shows a sample row in the UserAccessAnalytics table, for the user Alex Johnson. Source entity is the user or service principal account, and target entity is the resource that the source entity has access to. The values of access level and access type depend on the access-control model of the target entity. You can see that Alex has Contributor access to the Azure subscription Contoso Hotels Tenant. The access control model of the subscription is Azure RBAC.
You can use the Jupyter notebook (the same notebook mentioned above) from the Azure Sentinel GitHub repository to visualize the permission analytics data. For detailed instructions on how to use the notebook, see the Guided Analysis - User Security Metadata notebook.
Hunting queries and exploration queries
Azure Sentinel provides out-of-the-box a set of hunting queries, exploration queries, and a workbook, based on the BehaviorAnalytics table. These tools present enriched data, focused on specific use cases, that indicate anomalous behavior.
Learn more about hunting and the investigation graph in Azure Sentinel.
In this document, you learned about Azure Sentinel's entity behavior analytics capabilities. For practical guidance on implementation, and to use the insights you've gained, see the following articles: