Microsoft 365 audit log collection

Audit logs serve an important role in maintaining, troubleshooting, and protecting both customer tenants and the internal Microsoft 365 infrastructure. Due to the scale at which Microsoft 365 operates, the collection and processing of audit logs must be strategically managed to ensure efficient and effective monitoring. This article provides an overview of Microsoft 365’s audit logging practices including what event types are captured, what information is contained in each log, how logging policies are enforced, and how log data is protected at all stages of its lifecycle.

Defining auditable events

The Microsoft 365 Security team is responsible for defining the baseline logs that must be collected across Microsoft 365. They maintain an official list of event types that each system must capture as well as the data each logged event must contain.

Microsoft 365 captures log data from various sources, including:

• Event logs
• AppLocker logs
• Performance data
• System Center data
• Call detail records
• Quality of experience data
• IIS Web Server logs
• SQL Server logs
• Syslog data
• Security audit logs

Each of the logs in this list must at least contain the following information to establish event type, source, location, outcome, time, and entity associated:

• Source User ID
• Target User ID – as relevant/appropriate for event type
• Event timestamp (Date & Time)
• Event details
  • Type
  • Outcome (success/failure)
  • Detail information – defined per event type
• Source & Target Hostname – as relevant/appropriate for event type
• Source & Target network addresses and protocols – as relevant/appropriate for event type

The list of auditable events and related data is informed by ongoing risk assessments, Microsoft 365 security standards, business requirements, and compliance requirements. The Microsoft 365 Security team reviews and updates the list of auditable events to account for new threats, system changes, lessons learned from past incidents, and changing compliance requirements.

Service teams can define additional logging requirements for their service in addition to the list of auditable events defined by the Microsoft 365 Security team. Application-specific events are reviewed and updated during service reviews and the planning phases of feature milestones. The Microsoft 365 Security team also helps guide these individual service teams on audit functions to meet their specific needs. Service-level auditable events are reviewed and updated whenever a significant change to the system is made.

Due to Microsoft’s scale, the amount of data captured must be balanced with the ability to store and process it. Collecting information on every possible event and the contents therein would be impractical from both a resource and analytical standpoint. By being selective with the kinds of log data collected, Microsoft can maintain the health and security of its information systems in an efficient and effective manner.

Centralized management

Microsoft 365 enforces logging requirements centrally through the policies set by the Microsoft 365 Security Team. Each Microsoft 365 system must include a custom logging agent, Office Data Loader (ODL), which is installed as part of the system baseline. ODL enforces central logging policies and is configured to collect and send the events defined by Microsoft 365 Security to centralized services for processing and storage.

ODL is configured to scrub all end-user information automatically and upload events in batches every five minutes. Any fields that contain customer data, such as tenant information and end-user identifiable information, are removed and replaced with a hash value. Permanent or irreversible changes to audit record content and time ordering, aside from the scrubbing described previously, is prohibited.

Log data is uploaded to a proprietary security monitoring solution for near real-time (NRT) analysis, checking logs for potential security events and performance indicators. Logs are also uploaded to an internal, big data computing service (Azure Data Lake) for long-term storage. All log data transfers occur over a FIPS 140-2-validated TLS connection to ensure data confidentiality during transit.

Most audit log data stored in Azure Data Lake are kept for at least one year to support investigations of security incidents and to meet regulatory retention requirements. Service teams may select alternative retention periods of 90 days or longer for specific types of log data to support the needs of their applications. Microsoft 365 log retention and backup policies ensure log data is readily available for incident investigations, compliance reporting, and any other business requirements.

Access control

Microsoft performs extensive monitoring and auditing of all delegation, privileges, and operations that occur within Microsoft 365. Access to Microsoft 365 data stored in Azure Data Lake is restricted to authorized personnel, and all access control requests and approvals are captured for security event analysis. Microsoft reviews access levels in near real-time to ensure that its systems can only be accessed by users who have authorized business justifications and meet the eligibility requirements. All permitted access is traceable to a unique user.

Microsoft restricts the management of audit logs to a limited subset of Security Team members responsible for audit functionality. Microsoft’s internal access control philosophy revolves around the principles of Just-In-Time (JIT) and Just-Enough-Access (JEA). As such, the Security Team does not have standing administrative access to Azure Data Lake, and all changes are recorded and audited.