Hybrid Security Monitoring using Azure Security Center and Azure Sentinel

This reference architecture illustrates how to use Azure Security Center and Azure Sentinel to monitor the security configuration and telemetry of on-premises and Azure operating system workloads. This includes Azure Stack.

Diagram illustrating deployed Microsoft Monitoring Agent on on-premises systems as well as on Azure based virtual machines transferring data to Azure Security Center and Azure Sentinel

Download a Visio file of this architecture.

Typical uses for this architecture include:

  • Best practices for integrating on-premises security and telemetry monitoring with Azure-based workloads
  • How to integrate Azure Security Center with Azure Stack
  • How to integrate Azure Security Center with Azure Sentinel

Architecture

The architecture consists of the following components:

  • Azure Security Center. This is an advanced, unified security-management platform that Microsoft offers to all Azure subscribers. Security Center is segmented as a cloud security posture management (CSPM) and cloud workload protection platform (CWPP). CWPP is defined by workload-centric security protection solutions, which are typically agent-based. Azure Security Center provides threat protection for Azure workloads, both on-premises and in other clouds, including Windows and Linux virtual machines (VMs), containers, databases, and Internet of Things (IoT). When activated, the Log Analytics agent deploys automatically into Azure Virtual Machines. For on-premises Windows and Linux servers and VMs, you can manually deploy the agent, use your organization's deployment tool, such as Microsoft Endpoint Protection Manager, or utilize scripted deployment methods. Security Center begins assessing the security state of all your VMs, networks, applications, and data.
  • Azure Sentinel. Is a cloud-native Security Information and Event Management (SIEM) and security orchestration automated response (SOAR) solution that uses advanced AI and security analytics to help you detect, hunt, prevent, and respond to threats across your enterprise.
  • Azure Stack. Is a portfolio of products that extend Azure services and capabilities to your environment of choice, from the datacenter to edge locations and remote offices. Systems that you integrate with Azure Stack typically utilize racks of four to sixteen servers, built by trusted hardware partners and delivered straight to your datacenter.
  • Azure Monitor. Collects monitoring telemetry from a variety of on-premises and Azure sources. Management tools, such as those in Azure Security Center and Azure Automation, also push log data to Azure Monitor.
  • Log Analytics workspace. Azure Monitor stores log data in a Log Analytics workspace, which is a container that includes data and configuration information.
  • Log Analytics agent. The Log Analytics agent collects monitoring data from the guest operating system and VM workloads in Azure, other cloud providers, and on-premises. The Log Analytics Agent supports Proxy configuration and, typically in this scenario, a Microsoft Operations Management Suite (OMS) Gateway acts as proxy.
  • On-premises network. This is the firewall configured to support HTTPS egress from defined systems.
  • On-premises Windows and Linux systems. Systems with the Log Analytics Agent installed.
  • Azure Windows and Linux VMs. Systems on which the Azure Security Center monitoring agent is installed.

Recommendations

The following recommendations apply for most scenarios. Follow these recommendations unless you have a specific requirement that overrides them.

Azure Security Center upgrade

This reference architecture uses Azure Security Center to monitor on-premises systems, Azure VMs, Azure Monitor resources, and even VMs hosted by other cloud providers. To support that functionality, the standard fee-based tier of Azure Security Center is needed. We recommend that you use the 30-day free trial to validate your requirements.

Details about Azure Security Center pricing can be found here.

Customized Log Analytics Workspace

Azure Sentinel needs access to a Log Analytics workspace. In this scenario, you can’t use the default ASC Log Analytics workspace with Azure Sentinel. You’ll need to create a customized workspace. Data retention for a customized workspace is based on the workspace pricing tier, and you can find pricing models for Monitor Logs here.

Note

Azure Sentinel can run on workspaces in any general availability (GA) region of Log Analytics except the China and Germany (Sovereign) regions. Data that Azure Sentinel generates, such as incidents, bookmarks, and alert rules, which may contain some customer data sourced from these workspaces, is saved either in Europe (for Europe-based workspaces), in Australia (for Australia-based workspaces), or in the East US (for workspaces located in any other region).

Scalability considerations

The Log Analytics Agent for Windows and Linux is designed to have very minimal impact on the performance of VMs or physical systems.

Azure Security Center operational process won’t interfere with your normal operational procedures. Instead, it passively monitors your deployments and provides recommendations based on the security policies you enable.

Manageability considerations

Azure Security Center roles

Security Center assesses your resources’ configuration to identify security issues and vulnerabilities, and displays information related to a resource when you are assigned the role of owner, contributor, or reader for the subscription or resource group to which a resource belongs.

In addition to these roles, there are two specific Security Center roles:

  • Security Reader. A user that belongs to this role has read only rights to Security Center. The user can observe recommendations, alerts, a security policy, and security states, but can’t make changes.

  • Security Admin. A user that belongs to this role has the same rights as the Security Reader, and also can update security policies, and dismiss alerts and recommendations. Typically, these are users that manage the workload.

  • The security roles, Security Reader and Security Admin, have access only in Security Center. The security roles don’t have access to other Azure service areas, such as storage, web, mobile, or IoT.

Azure Sentinel subscription

  • To enable Azure Sentinel, you need contributor permissions to the subscription in which the Azure Sentinel workspace resides.
  • To use Azure Sentinel, you need contributor or reader permissions on the resource group to which the workspace belongs.
  • Azure Sentinel is a paid service. For more information, refer to Azure Sentinel pricing.

Security considerations

A security policy defines the set of controls that are recommended for resources within a specified subscription. In Azure Security Center, you define policies for your Azure subscriptions according to your company's security requirements and the type of applications or data sensitivity for each subscription.

The security policies that you enable in Azure Security Center drive security recommendations and monitoring. To learn more about security policies, refer to Strengthen your security policy with Azure Security Center. You can assign security policies in Azure Security Center only at the management or subscription group levels.

Note

Part one of the reference architecture details how to enable Azure Security Center to monitor Azure resources, on-premises systems, and Azure Stack systems.

Deploy the solution

Create a Log Analytics workspace in Azure Portal

  1. Sign into the Azure portal as a user with Security Admin privileges.
  2. In the Azure portal, select All services. In the list of resources, enter Log Analytics. As you begin entering, the list filters based on your input. Select Log Analytics workspaces.
  3. Select Add on the Log Analytics page.
  4. Provide a name for the new Log Analytics workspace, such as ASC-SentinelWorkspace. This name must be globally unique across all Azure Monitor subscriptions.
  5. Select a subscription by selecting from the drop-down list if the default selection is not appropriate.
  6. For Resource Group, choose to use an existing resource group or create a new one.
  7. For Location, select an available geolocation.
  8. Select OK to complete the configuration. New Workspace created for the architecture

Enable Security Center

While you're still signed into the Azure portal as a user with Security Admin privileges, select Security Center in the panel. Security Center - Overview opens:

Security Center Overview dashboard blade opens

Security Center automatically enables the Free tier for any of the Azure subscriptions not previously onboarded by you or another subscription user.

Upgrade to the Standard tier

Important

This reference architecture uses the 30-day free trial of Security Center Standard tier.

  1. On the Security Center main menu, select Getting Started.
  2. Select the Upgrade Now button. Security Center lists your subscriptions and workspaces that are eligible for use in the Standard tier.
  3. You can select eligible workspaces and subscriptions to start your trial. Select the previously created workspace, ASC-SentinelWorkspace. from the drop-down menu.
  4. In the Security Center main menu, select Start trial.
  5. The Install Agents dialog box should display.
  6. Select the Install Agents button. The Security Center - Coverage blade displays and you should observe your selected subscription in the Standard coverage tab. Security Coverage blade showing your subscriptions should be open

You've now enabled automatic provisioning and Security Center will install the Log Analytics Agent for Windows (HealthService.exe) and the omsagent for Linux on all supported Azure VMs and any new ones that you create. You can turn off this policy and manually manage it, although we strongly recommend automatic provisioning.

To learn more about the specific Security Center features available in Windows and Linux, refer to Feature coverage for machines.

Enable Azure Security Center monitoring of on-premises Windows computers

  1. In the Azure Portal on the Security Center - Overview blade, select the Get Started tab.
  2. Select Configure under Add new non-Azure computers. A list of your Log Analytics workspaces displays, and should include the ASC-SentinelWorkspace.
  3. Select this workspace. The Direct Agent blade opens with a link for downloading a Windows agent and keys for your workspace identification (ID) to use when you configure the agent.
  4. Select the Download Windows Agent link applicable to your computer processor type to download the setup file.
  5. To the right of Workspace ID, select Copy, and then paste the ID into Notepad.
  6. To the right of Primary Key, select Copy, and then paste the key into Notepad.

Install the Windows agent

To install the agent on the targeted computers, follow these steps.

  1. Copy the file to the target computer and then Run Setup.
  2. On the Welcome page, select Next.
  3. On the License Terms page, read the license and then select I Agree.
  4. On the Destination Folder page, change or keep the default installation folder and then select Next.
  5. On the Agent Setup Options page, choose to connect the agent to Azure Log Analytics and then select Next.
  6. On the Azure Log Analytics page, paste the Workspace ID and Workspace Key (Primary Key) that you copied into Notepad in the previous procedure.
  7. If the computer should report to a Log Analytics workspace in Azure Government cloud, select Azure US Government from the Azure Cloud drop-down list. If the computer needs to communicate through a proxy server to the Log Analytics service, select Advanced, and then provide the proxy server's URL and port number.
  8. After you provide the necessary configuration settings, select Next. Log Analytics Agent setup page for connecting agent to an Azure Log Analytics workspace
  9. On the Ready to Install page, review your choices and then select Install.
  10. On the Configuration completed successfully page, select Finish.

When complete, the Log Analytics agent appears in Windows Control Panel, and you can review your configuration and verify that the agent is connected.

For further information about installing and configuring the agent, refer to Install Log Analytics agent on Windows computers.

The Log Analytics Agent service collects event and performance data, executes tasks, and other workflows defined in a management pack. Security Center extends its cloud workload protection platforms by integrating with Microsoft Defender Advanced Threat Protection (ATP) for Servers. Together, they provide comprehensive endpoint detection and response (EDR) capabilities.

For more information about Microsoft Defender ATP, refer to Onboard servers to the Microsoft Defender ATP service.

Enable Azure Security Center monitoring of on-premises Linux computers

  1. Return to the Getting Started tab as previously described.
  2. Select Configure under Add new non-Azure computers. A list of your Log Analytics workspaces displays. The list should include the ASC-SentinelWorkspace that you created.
  3. On the Direct Agent blade under DOWNLOAD AND ONBOARD AGENT FOR LINUX, select copy to copy the wget command.
  4. Open Notepad and then paste this command. Save this file to a location that you can access from your Linux computer.

Note

On Unix and Linux operating systems, wget is a tool for non-interactive file downloading from the web. It supports HTTPS, FTPs, and proxies.

The Linux agent uses the Linux Audit Daemon framework. Security Center integrates functionalities from this framework within the Log Analytics agent, which enables audit records to be collected, enriched, and aggregated into events by using the Log Analytics Agent for Linux. Security Center continuously adds new analytics that use Linux signals to detect malicious behaviors on cloud and on-premises Linux machines.

For a list of the Linux alerts, refer to the Reference table of alerts.

Install the Linux agent

To install the agent on the targeted Linux computers, follow these steps:

  1. On your Linux computer, open the file that you previously saved. Select and copy the entire content, open a terminal console, and then paste the command.
  2. Once the installation finishes, you can validate that the omsagent is installed by running the pgrep command. The command will return the omsagent process identifier (PID). You can find the logs for the agent at: /var/opt/microsoft/omsagent/"workspace id"/log/.

It can take up to 30 minutes for the new Linux computer to display in Security Center.

Enable Azure Security Center monitoring of Azure Stack VMs

After you onboard your Azure subscription, you can enable Security Center to protect your VMs running on Azure Stack by adding the Azure Monitor, Update and Configuration Management VM extension from the Azure Stack marketplace. To do this:

  1. Return to the Getting Started tab as previously described.
  2. Select Configure under Add new non-Azure computers. A list of your Log Analytics workspaces displays, and it should include the ASC-SentinelWorkspace that you created.
  3. On the Direct Agent blade there is a link for downloading the agent and keys for your workspace ID to use during agent configuration. You don’t need to download the agent manually. It’ll be installed as a VM extension in the following steps.
  4. To the right of Workspace ID, select Copy, and then paste the ID into Notepad.
  5. To the right of Primary Key, select Copy, and then paste the key into Notepad.

Enable ASC monitoring of Azure Stack VMs

Azure Security Center uses the Azure Monitor, Update and Configuration Management VM extension bundled with Azure Stack. To enable the Azure Monitor, Update and Configuration Management extension, follow these steps:

  1. In a new browser tab, sign into your Azure Stack portal.
  2. Refer to the Virtual machines page, and then select the virtual machine that you want to protect with Security Center.
  3. Select Extensions. The list of VM extensions installed on this VM displays.
  4. Select the Add tab. The New Resource menu blade opens and displays the list of available VM extensions.
  5. Select the Azure Monitor, Update and Configuration Management extension and then select Create. The Install extension configuration blade opens.
  6. On the Install extension configuration blade, paste the Workspace ID and Workspace Key (Primary Key) that you copied into Notepad in the previous procedure.
  7. When you finish providing the necessary configuration settings, select OK.
  8. Once the extension installation completes, its status will display as Provisioning Succeeded. It might take up to one hour for the VM to appear in the Security Center portal.

For more information about installing and configuring the agent for Windows, refer to Install the agent using setup wizard.

For troubleshooting issues for the Linux agent, refer to How to troubleshoot issues with the Log Analytics agent for Linux.

Now you can monitor your Azure VMs and non-Azure computers in one place. Azure Compute provides you with an overview of all VMs and computers along with recommendations. Each column represents one set of recommendations, and the color represents the VMs or computers and the current security state for that recommendation. Security Center also provides any detections for these computers in security alerts. ASC list of systems monitored on the Compute blade

There are two types of icons represented on the Compute blade:

Purple computer icon that represents a non-azure monitored computer Non-Azure computer

Blue terminal icon that represents a Azure monitored computer Azure computer

Note

Part two of the reference architecture will connect alerts from Azure Security Center and stream them into Azure Sentinel.

The role of Azure Sentinel is to ingest data from different data sources and perform data correlation across these data sources. Azure Sentinel leverages machine learning and AI to make threat hunting, alert detection, and threat responses smarter.

To onboard Azure Sentinel, you need to enable it, and then connect your data sources. Azure Sentinel comes with a number of connectors for Microsoft solutions, which are available out of the box and provide real-time integration, including Microsoft Security Center, Microsoft Threat Protection solutions, Microsoft 365 sources (including Office 365), Azure Active Directory (Azure AD), Azure ATP, Microsoft Cloud App Security, and more. Additionally, there are built-in connectors to the broader security ecosystem for non-Microsoft solutions. You can also use Common Event Format, syslog, or the Representational State Transfer API to connect your data sources with Azure Sentinel.

Requirements for integrating Azure Sentinel with Azure Security Center

  1. A Microsoft Azure Subscription
  2. A Log Analytics workspace that isn't the default workspace created when you enable Azure Security Center.
  3. Azure Security Center with Security Center Standard tier enabled.

All three requirements should be in place if you worked through the previous section.

Global prerequisites

  • To enable Azure Sentinel, you need contributor permissions to the subscription in which the Azure Sentinel workspace resides.
  • To use Azure Sentinel, you need contributor or reader permissions on the resource group to which the workspace belongs.
  • You might need additional permissions to connect specific data sources. You don't need additional permissions to connect to ASC.
  • Azure Sentinel is a paid service. For more information, refer to Azure Sentinel pricing.

Enable Azure Sentinel

  1. Sign into the Azure portal with a user that has contributor rights for ASC-Sentinelworkspace.
  2. Search for and select Azure Sentinel. In the Azure portal search for the term "Azure Sentinel"
  3. Select Add.
  4. On the Azure Sentinel blade, select ASC-Sentinelworkspace.
  5. In Azure Sentinel, select Data connectors from the navigation menu.
  6. From the data connectors gallery, select Azure Security Center, and select the Open connector page button. In Azure Sentinel showing the open Collectors page
  7. Under Configuration, select Connect next to those subscriptions for which you want alerts to stream into Azure Sentinel. The Connect button will be available only if you have the required permissions and the ASC Standard tier subscription.
  8. You should now observe the Connection Status as Connecting. After connecting, it will switch to Connected.
  9. After confirming the connectivity, you can close ASC Data Connector settings and refresh the page to observe alerts in Azure Sentinel. It might take some time for the logs to start syncing with Azure Sentinel. After you connect, you'll observe a data summary in the Data received graph and the connectivity status of the data types.
  10. You can select whether you want the alerts from Azure Security Center to automatically generate incidents in Azure Sentinel. Under Create incidents, select Enabled to turn on the default analytics rule that automatically creates incidents from alerts. You can then edit this rule under Analytics, in the Active rules tab.
  11. To use the relevant schema in Log Analytics for the Azure Security Center alerts, search for SecurityAlert.

One advantage of using Azure Sentinel as your SIEM is that it provides data correlation across multiple sources, which enables you to have an end-to-end visibility of your organization’s security-related events.

Note

To learn how to increase visibility in your data and identify potential threats, refer to Azure playbooks on TechNet Gallery, which has a collection of resources including a lab in which you can simulate attacks. You should not use this lab in a production environment.

To learn more about Azure Sentinel, refer to the following articles:

Cost considerations

References

Azure Monitor

Azure Security Center

Azure Sentinel

Azure Stack