System monitoring for security in Azure Australia

Having robust security strategies that include real-time monitoring and routine security assessments are critical for you to enhance the day to day operational security of your IT environments, including cloud.

Cloud security is a joint effort between the customer and the cloud provider. There are four services which Microsoft Azure provides to facilitate these requirements with consideration to the recommendations contained within the Australian Cyber Security Centre's (ACSC) Information Security Manual Controls (ISM), specifically, the implementation of centralised event logging, event log auditing, and security vulnerability assessment and management. The Microsoft Azure services are:

  • Azure Security Center
  • Azure Monitor
  • Azure Advisor
  • Azure Policy

The ACSC recommends that you use these services for PROTECTED data. By using these services, you can proactively monitor and analyse your IT environments, and make informed decisions on where to best allocate resources to enhance your security. Each of these services is part of a combined solution to provide you with the best insight, recommendations, and protection possible.

Azure Security Center

Azure Security Center provides a unified security management console that you use to monitor and enhance the security of Azure resources and your hosted data. Azure Security Center includes Secure Score, a score based on an analysis of the state of best practice configuration from Azure Advisor and the overall compliance of Azure Policy.

Azure Security Center provides Azure customers with the following features:

  • Security policy, assessment, and recommendations
  • Security event collection and search
  • Access and application controls
  • Advanced Threat Detection
  • Just-in-time Virtual Machines access control
  • Hybrid Security

The scope of resources monitored by Azure Security Center can be expanded to include supported on-premises resources in a hybrid-cloud environment. This includes on-premises resources currently being monitored by a supported version of System Center Operations Manager.

The Security Center "Standard" tier also provides cloud-based security controls required by the ASD Essential 8. These include application filtering and restriction of administrative privilege via just-in-time access.

Azure Monitor

Azure Monitor is the centralised logging solution for all Azure Resources, and includes Log Analytics and Application Insights. Two key data types are collected from Azure resources: logs and metrics. Once collected in Azure Monitor, logging information can be used by a wide range of tools and for a variety of purposes.

Azure Monitor Overview

Azure Monitor also includes the "Azure Activity Log". The SActivity Log stores all subscription level events that have occurred within Azure. It allows Azure customers to see the "who, what and when" behind operations undertaken on their Azure resources. Both resource based logging sent to Azure Monitor and Azure Activity Log events can be analysed using the in-built Kusto query language. These logs can then be exported, used to create custom dashboards and views, and configured to trigger alerts and notifications.

Azure Advisor

Azure Advisor analyses supported Azure resources, system-generated log files, and current resource configurations within your Azure subscription. The analysis provided in Azure Advisor is generated in real time and based upon Microsoft's recommended best practices. Any supported Azure resources added to your environment will be analysed and appropriate recommendations will be provided. Azure Advisor recommendations are categorised into four best practice categories:

  • Security
  • High Availability
  • Performance
  • Cost

Security recommendations generated by Azure Advisor form part of the overall security analysis provided by Azure Security Center.

The information gathered by Azure Advisor provides administrators with:

  • Insight into resource configuration that does not meet recommended best practice
  • Guidance on specific remediation actions to undertake
  • Rankings indicating which remediation actions should be undertaken as a high priority

Azure Policy

Azure Policy provides the ability to apply rules that govern the types of Azure resources and their allowed configuration. Policy can be used to control resource creation and configuration, or it can be used to audit configuration settings across an environment. These audit results can be used to form the basis of remediation activities. Azure Policy differs from Azure role-based access control (Azure RBAC); Azure Policy is used to restrict resources and their configuration, Azure RBAC is used to restrict privileged access to Azure users.

Whether the specific policy is being enforced or the effect of the policy is being audited, policy compliance is continually monitored, and overall and resource-specific compliance information is provided to administrators. Azure Policy compliance data is provided to Azure Security Center and forms part of the Secure Score.

Key design considerations

When implementing an event log strategy, the ACSC ISM highlights the following considerations:

  • Centralised logging facilities
  • Specific events to be logged
  • Event log protection
  • Event log retention
  • Event log auditing

In additional to collecting and managing logs, the ISM also recommends routine vulnerability assessment of an organisation's IT environment.

Centralised logging

Any logging solution should, wherever possible, consolidate captured logs into a single data repository. This not only reduces operational complexity and prevents the creation of multiple data silos, it enables data collected from multiple sources to be analysed together allowing any correlating events to be identified. This is critical for detecting and managing the scope of any cyber security incidents.

This requirement is met for all Azure customers with Azure Monitor. This offering not only provides a centralised logging repository in Azure for all Azure resources, it also enables you to stream your data to an Azure Event Hub. Azure Event Hubs provides a fully managed, real-time data ingestion service. Once Azure Monitor data is streamed to an Azure Event Hub, the data can also be easily connected to existing supported Security information and event management (SIEM) repositories and additional third party monitoring tools.

Microsoft also offers its own Azure native SIEM solution, Azure Sentinel. Azure Sentinel supports a wide variety of data connectors and can be used to monitor security events across an entire enterprise. By combining the data from supported data connectors, Azure Sentinel's built-in machine learning, and the Kusto query language, security administrators are provided with a single solution for alert detection, threat visibility, proactive hunting, and threat response. Azure Sentinel also provides a hunting and notebook feature that allows security administrators to record all the steps undertaken as part of a security investigation in a reuseable playbook that can be shared within an organisation. Security Administrators can even use the built-in User Analytics to investigate the actions of a single nominated user.

Logged events and log detail

The ISM provides a detailed list of event log types that should be included in any logging strategy. Any captured logs must contain sufficient detail to be of any practical use in conducting analysis and investigations.

The logs collected in Azure fall under one of following three categories:

  • Control and Management Logs: These logs provide information about Azure Resource Manager CREATE, UPDATE, and DELETE operations.

  • Data Plane Logs: These contain events raised as part of Azure resource usage. Includes sources such as Windows event logs including System, Security, and Application logs.

  • Processed Events: These events contain information about events and alerts that have been automatically processed on the customer's behalf by Azure. An example of a Processed Event is an Azure Security Center Alert.

Azure virtual machine monitoring is enhanced by the deployment of the virtual machine agent for both Windows and Linux. This markedly increases the breadth of logging information gathered. Deployment of this agent can be configured to occur automatically via the Azure Security Center.

Microsoft provides detailed information about Azure resource-specific logs and their schemas.

Log retention and protection

Event logs must be stored securely for the required retention period. The ISM advises that logs are retained for a minimum of seven years. Azure provides a number of means to ensure the long life of your collected logs. By default, the Azure Log events are stored for 90 days. Log data captured by Azure Monitor can be moved and stored on an Azure Storage account as required for long-term retention. Activity logs stored on an Azure Storage Account can be retained for a set number of days, or indefinitely if necessary.

Azure Storage Accounts used to store Azure Log events can be made geo-redundant and can be backed up using Azure Backup. Once captured by Azure Backup, any deletion of backups containing logs requires administrative approval and backups marked for deletion are still held for 14 days allowing for recovery. Azure Backup allows for 9999 copies of a protected instance, providing over 27 years of daily backups.

Azure role-based access control (Azure RBAC) should be used to control access to resources used for Azure logging. Azure Monitor, Azure Storage accounts, and Azure Backups should be configured with Azure RBAC to ensure the security of the data contained within the logs.

Log auditing

The true value of logs is realised once they are analysed. Using both automated and manual analysis, and being familiar with the available tools, will assist you to detect and manage breaches of organisational security policy, and cyber security incidents. Azure Monitor provides a rich set of tools to analyse collected logs. The result of this analysis can then be shared between systems, visualised, or disseminated in multiple formats.

Log data stored in Azure Monitor is kept in a Log Analytics Workspace. All analysis begins with a query. Azure Monitor queries are written in the Kusto query language. Queries form the basis of all outputs from Azure Monitor, from Azure Dashboards to Alert Rules.

Azure Log Queries Overview

Auditing of logs can be enhanced through the use of Monitoring Solutions. These are pre-packaged solutions that contain collection logic, queries, and data visualisation views. Microsoft provide a number of Monitoring Solutions and additional solutions from product vendors can be found in the Azure Marketplace.

Vulnerability assessment and management

The ISM notes that routine vulnerability assessment and management are essential. Your IT environment is constantly evolving, and the external security threat is endlessly changing. With Azure Security Center you can do automated vulnerability assessments and get guidance on how to plan and perform remediation activities.

Secure Score in Azure Security Center gives you a list of recommendations that, when applied, will improve the security of your environment. The list is sorted by the impact on the overall Secure Score from highest to lowest. Ordering the list by impact allows you to focus on the highest priority recommendations that present the most value in enhancing your security.

Azure Policy also plays a key part in the ongoing vulnerability assessment. The types of policy available in Azure Policy range from enforcing resource tags and values, to restricting the Azure regions in which resources can be created, to blocking the creation of particular resource types altogether. A set of Azure policies can be grouped into Initiatives. Initiatives are used to apply related Azure policies that, when applied together as a group, form the basis of a specific security or compliance objective.

Azure Policy has a library of built-in policy definitions which is constantly growing. Azure portal also gives you the option to author your own custom Azure Policy definitions. Once you find a policy in the existing library or create a new one, you can then assign the policy to Azure resources. These assignments can be scoped at various levels in the resource management hierarchy. Policy assignment is inherited, meaning all child resources within a scope receive the same policy assignment. Resources can also be excluded from scoped policy assignment as required.

All deployed Azure policies contribute to an organisation's Secure Score. In a highly bespoke environment, custom Azure Policy definitions can be created and deployed to provide audit information tailored to specific workloads.

Getting started

To start with Azure Security Center and make full use of Azure Monitor, Advisor and Policy, Microsoft recommends the following initial steps:

  • Enable Azure Security Center
  • Upgrade to the Standard Tier
  • Enable Automatic Provisioning of the Microsoft Monitoring Agent to supported Azure Virtual Machines
  • Review, prioritise, and mitigate the security recommendations and alerts on the Security Centre dashboard

Next steps

Read Azure Policy and Azure Blueprints for details on implementing governance and control over your Azure Australia resources to ensure policy and regulatory compliance.