Security logging and analysis options in Azure
Today we have a great blog post for those of you interested in the security logging options in Microsoft Azure. Sanjay Narang was the thought leader on this article, with Becky Ochs, Namgyal Dolker and I working together with him. Enjoy!
We’ve seen that many of you are looking for a way to collect and analyze security (and other) logs in Azure. Analysis of these logs can provide you with a ton of useful information and insight into your current security posture. Armed with this information, you’ll be able to detect security issues in your Azure environment and be able to take action based on that information.
For example, you can use these logs to:
- Detect threats and policy violations
- Help achieve regulatory compliance
- Search for anomalous network, host, or user activity
Security Information and Event Management (SIEM) systems are used to collect and analyze logs and many of you are using them today to help you with your log collection and analysis. However, to get the most out of your SIEM systems, you must be able to get the logging information you need into them, and that means knowing what logging information is available to you and how you can access it.
To help you solve this problem, this blog post will provide you with a summary of the different types of logs that are available today in Azure and describe how to access those logs. We’ll also provide links to REST APIs for accessing each type of log. REST APIs enable you to import these logs into most SIEM engines.
Subscription Management Level Logs
Azure can log the events that impact your entire subscription or you can monitor events that impact specific resources.
Event Services Logs
Azure provides the Azure Insights REST API as well as .Net Library APIs. You can use these APIs to get visibility into your Azure resources. All task operations conform to the HTTP/1.1 protocol specification and each operation returns an x-ms-request-id header that can be used to obtain information about the request.
A key part of these APIs is Event Services. Event Services enables you to obtain information about events (or operations) related to your Azure resources. Logs of all these events/operations are exposed through the well-documented Management Events REST API for the subscription. The Management Events Rest API logs all Create/Update/Delete operations, as well as privileged Read operations.
Event Services is used by portals to enable you to easily view logs in a user interface. It is also consumed by other APIs, such as the Azure Resource Manager (ARM) API.
- Operations Logs: these are available in the v1 portal under management services. You can also find them through the service management Operations REST API.
- Audit Logs: these are available in the new portal and through PowerShell commands such as Get-AzureSubscriptionIdLog or Get-AzureResourceProviderLog.
Service Level Logs
In addition to subscription level logs, many services provide more detailed logs.
Diagnostics logs can be used for Virtual Machines and Cloud Services. These logs include:
- Event Logs
- Custom Application Logs
- IIS Logs
- Crash Dumps
- Performance Counters
All of these logs are described in detail in the article Microsoft Azure Security and Audit Log Management.
Since diagnostics logs are stored in Azure Storage, you can use Storage Service REST API to pull these logs into most SIEM engines. The collection can be scheduled at regular intervals by configuring the ScheduledTransferPeriod Property. The minimum transfer period you can specify is 1 minute.
Azure AD Reports
Azure AD Reports provide a variety of activity, security and audit reports. You can use this information to determine where security risks may lie. This helps guide your security investigations and subsequent mitigations. Reports are also available through the REST API.
For more information on what is included in the audit reports, see Azure Active Directory Audit Report Events.
Azure Storage Analytics
Azure Storage Analytics provides logs (traces of executed requests for Blobs, Tables and Queues) and metrics (summary of key capacity and request statistics). Logs can be fetched using the Storage Service REST API.
From a security perspective, Azure Storage Analytics allows you to run the following the types of analyses (more details here):
- How many anonymous requests is my application seeing from a given range of IP addresses?
- Which containers are being accessed the most?
- How many times is a particular SAS URL being accessed and how?
- Who issued the request to delete a container?
Azure SQL Database Auditing
Azure SQL Database Auditing tracks database events and writes audited events to an audit log in your Azure Storage account. More details are available here. These logs can be fetched using Storage Service REST API.
The article Microsoft Azure Security and Audit Log Management provides detailed information about logging and audit management in Azure. We highly recommend that you read the article to get further insights into much of what we covered in this blog post.
We hope that you find these logging and auditing resources useful. If you have questions, comments, or suggestions, please let us know! Just enter a comment at the bottom of the blog and we’ll answer your questions as quickly as possible.