Azure security baseline for Azure Databricks

This security baseline applies guidance from the Azure Security Benchmark version 2.0 to Azure Databricks. The Azure Security Benchmark provides recommendations on how you can secure your cloud solutions on Azure. The content is grouped by the security controls defined by the Azure Security Benchmark and the related guidance applicable to Azure Databricks.

Note

Controls not applicable to Azure Databricks, and those for which the global guidance is recommended verbatim, have been excluded. To see how Azure Databricks completely maps to the Azure Security Benchmark, see the full Azure Databricks security baseline mapping file.

Network Security

For more information, see the Azure Security Benchmark: Network Security.

NS-1: Implement security for internal traffic

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

NS-2: Connect private networks together

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

NS-3: Establish private network access to Azure services

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

NS-4: Protect applications and services from external network attacks

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

NS-5: Deploy intrusion detection/intrusion prevention systems (IDS/IPS)

Guidance: Use Azure Firewall threat intelligence-based filtering to alert on and/or block traffic to and from known malicious IP addresses and domains. The IP addresses and domains are sourced from the Microsoft Threat Intelligence feed. When payload inspection is required, you can deploy a third-party intrusion detection/intrusion prevent system (IDS/IPS) from Azure Marketplace with payload inspection capabilities. Alternately you can use host-based IDS/IPS or a host-based endpoint detection and response (EDR) solution in conjunction with or instead of network-based IDS/IPS.

Note: If you have a regulatory or other requirement for IDS/IPS use, ensure that it is always tuned to provide high-quality alerts to your SIEM solution.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

NS-6: Simplify network security rules

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

NS-7: Secure Domain Name Service (DNS)

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Identity Management

For more information, see the Azure Security Benchmark: Identity Management.

IM-1: Standardize Azure Active Directory as the central identity and authentication system

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

IM-2: Manage application identities securely and automatically

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

IM-3: Use Azure AD single sign-on (SSO) for application access

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

IM-4: Use strong authentication controls for all Azure Active Directory based access

Guidance: Azure AD supports strong authentication controls through multi-factor authentication (MFA) and strong passwordless methods.

  • Multi-factor authentication: Enable Azure AD MFA and follow Microsoft Defender for Cloud identity and access management recommendations for your MFA setup. MFA can be enforced on all users, select users, or at the per-user level based on sign-in conditions and risk factors.

  • Passwordless authentication: Three passwordless authentication options are available: Windows Hello for Business, Microsoft Authenticator app, and on-premises authentication methods such as smart cards.

For administrator and privileged users, ensure the highest level of the strong authentication method is used, followed by rolling out the appropriate strong authentication policy to other users.

If legacy password-based authentication is still used for Azure AD authentication, please be aware that cloud-only accounts (user accounts created directly in Azure) have a default baseline password policy. And hybrid accounts (user accounts that come from on-premises Active Directory) follow the on-premises password policies. When using password-based authentication, Azure AD provides a password protection capability that prevents users from setting passwords that are easy to guess. Microsoft provides a global list of banned passwords that is updated based on telemetry, and customers can augment the list based on their needs (e.g. branding, cultural references, etc.). This password protection can be used for cloud-only and hybrid accounts.

Note: Authentication based on password credentials alone is susceptible to popular attack methods. For higher security, use strong authentication such as MFA and a strong password policy. For third-party applications and marketplace services that may have default passwords, you should change them during initial service setup.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IM-5: Monitor and alert on account anomalies

Guidance: Azure AD provides the following data sources:

  • Sign-ins – The sign-ins report provides information about the usage of managed applications and user sign-in activities.

  • Audit logs - Provides traceability through logs for all changes made through various features in Azure AD. Examples of logged changes audit logs include adding or removing users, apps, groups, roles, and policies.

  • Risky sign-ins - A risky sign-in is an indicator for a sign-in attempt that might have been performed by someone who is not the legitimate owner of a user account.

  • Users flagged for risk - A risky user is an indicator for a user account that might have been compromised.

These data sources can be integrated with Azure Monitor, Microsoft Sentinel or third party SIEM systems.

Microsoft Defender for Cloud can also alert on certain suspicious activities such as an excessive number of failed authentication attempts, and deprecated accounts in the subscription.

Azure Advanced Threat Protection (ATP) is a security solution that can use on-premises Active Directory signals to identify, detect, and investigate advanced threats, compromised identities, and malicious insider actions.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IM-6: Restrict Azure resource access based on conditions

Guidance: Use Azure AD conditional access for more granular access control based on user-defined conditions, such as requiring user logins from certain IP ranges to use MFA. A granular authentication session management can also be used through Azure AD conditional access policy for different use cases.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IM-7: Eliminate unintended credential exposure

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

IM-8: Secure user access to legacy applications

Guidance: Ensure you have modern access controls and session monitoring for legacy applications and the data they store and process. While VPNs are commonly used to access legacy applications, they often have only basic access control and limited session monitoring.

Azure AD Application Proxy enables you to publish legacy on-premises applications to remote users with single sign-on (SSO) while explicitly validating the trustworthiness of both remote users and devices with Azure AD Conditional Access.

Alternatively, Microsoft Defender for Cloud Apps is a cloud access security broker (CASB) service that can provide controls for monitoring a user’s application sessions and blocking actions (for both legacy on-premises applications and cloud software as a service (SaaS) applications).

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Privileged Access

For more information, see the Azure Security Benchmark: Privileged Access.

PA-1: Protect and limit highly privileged users

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PA-2: Restrict administrative access to business-critical systems

Guidance: Isolate access to business-critical systems by restricting which accounts are granted privileged access to the subscriptions and management groups they are in. Ensure that you also restrict access to the management, identity, and security systems that have administrative access to your business critical assets, such as Active Directory Domain Controllers (DCs), security tools, and system management tools with agents installed on business critical systems. Attackers who compromise these management and security systems can immediately weaponize them to compromise business critical assets.

All types of access controls should be aligned to your enterprise segmentation strategy to ensure consistent access control.

Ensure to assign separate privileged accounts that are distinct from the standard user accounts used for email, browsing, and productivity tasks.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

PA-3: Review and reconcile user access regularly

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PA-4: Set up emergency access in Azure AD

Guidance: To prevent being accidentally locked out of your Azure AD organization, set up an emergency access account for access when normal administrative accounts cannot be used. Emergency access accounts are usually highly privileged, and they should not be assigned to specific individuals. Emergency access accounts are limited to emergency or "break glass"' scenarios where normal administrative accounts can't be used. You should ensure that the credentials (such as password, certificate, or smart card) for emergency access accounts are kept secure and known only to individuals who are authorized to use them only in an emergency.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

PA-5: Automate entitlement management

Guidance: Use Azure AD entitlement management features to automate access request workflows, including access assignments, reviews, and expiration. Dual or multi-stage approval is also supported.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

PA-6: Use privileged access workstations

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PA-7: Follow just enough administration (least privilege principle)

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PA-8: Choose approval process for Microsoft support

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Data Protection

For more information, see the Azure Security Benchmark: Data Protection.

DP-1: Discovery, classify and label sensitive data

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

DP-2: Protect sensitive data

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

DP-3: Monitor for unauthorized transfer of sensitive data

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

DP-4: Encrypt sensitive information in transit

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

DP-5: Encrypt sensitive data at rest

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Asset Management

For more information, see the Azure Security Benchmark: Asset Management.

AM-1: Ensure security team has visibility into risks for assets

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

AM-2: Ensure security team has access to asset inventory and metadata

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

AM-3: Use only approved Azure services

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

AM-4: Ensure security of asset lifecycle management

Guidance: Establish or update security policies that address asset lifecycle management processes for potentially high impact modifications. These modifications include changes to: identity providers and access, data sensitivity, network configuration, and administrative privilege assignment.

Remove Azure resources when they are no longer needed.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

AM-5: Limit users' ability to interact with Azure Resource Manager

Guidance: Use Azure AD Conditional Access to limit users' ability to interact with Azure Resource Manager by configuring "Block access" for the "Microsoft Azure Management" App.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

AM-6: Use only approved applications in compute resources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Logging and Threat Detection

For more information, see the Azure Security Benchmark: Logging and Threat Detection.

LT-1: Enable threat detection for Azure resources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-2: Enable threat detection for Azure identity and access management

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-3: Enable logging for Azure network activities

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-4: Enable logging for Azure resources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-5: Centralize security log management and analysis

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-6: Configure log storage retention

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

LT-7: Use approved time synchronization sources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Incident Response

For more information, see the Azure Security Benchmark: Incident Response.

IR-1: Preparation – update incident response process for Azure

Guidance: Ensure your organization has processes to respond to security incidents, has updated these processes for Azure, and is regularly exercising them to ensure readiness.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IR-2: Preparation – setup incident notification

Guidance: Set up security incident contact information in Microsoft Defender for Cloud. This contact information is used by Microsoft to contact you if the Microsoft Security Response Center (MSRC) discovers that your data has been accessed by an unlawful or unauthorized party. You also have options to customize incident alert and notification in different Azure services based on your incident response needs.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IR-3: Detection and analysis – create incidents based on high quality alerts

Guidance: Ensure you have a process to create high-quality alerts and measure the quality of alerts. This allows you to learn lessons from past incidents and prioritize alerts for analysts, so they don’t waste time on false positives.

High-quality alerts can be built based on experience from past incidents, validated community sources, and tools designed to generate and clean up alerts by fusing and correlating diverse signal sources.

Microsoft Defender for Cloud provides high-quality alerts across many Azure assets. You can use the ASC data connector to stream the alerts to Microsoft Sentinel. Microsoft Sentinel lets you create advanced alert rules to generate incidents automatically for an investigation.

Export your Microsoft Defender for Cloud alerts and recommendations using the export feature to help identify risks to Azure resources. Export alerts and recommendations either manually or in an ongoing, continuous fashion.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IR-4: Detection and analysis – investigate an incident

Guidance: Ensure analysts can query and use diverse data sources as they investigate potential incidents, to build a full view of what happened. Diverse logs should be collected to track the activities of a potential attacker across the kill chain to avoid blind spots. You should also ensure insights and learnings are captured for other analysts and for future historical reference.

The data sources for investigation include the centralized logging sources that are already being collected from the in-scope services and running systems, but can also include:

  • Network data – use network security groups' flow logs, Azure Network Watcher, and Azure Monitor to capture network flow logs and other analytics information.

  • Snapshots of running systems:

    • Use Azure virtual machine's snapshot capability to create a snapshot of the running system's disk.

    • Use the operating system's native memory dump capability to create a snapshot of the running system's memory.

    • Use the snapshot feature of the Azure services or your software's own capability to create snapshots of the running systems.

Microsoft Sentinel provides extensive data analytics across virtually any log source and a case management portal to manage the full lifecycle of incidents. Intelligence information during an investigation can be associated with an incident for tracking and reporting purposes.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IR-5: Detection and analysis – prioritize incidents

Guidance: Provide context to analysts on which incidents to focus on first based on alert severity and asset sensitivity.

Microsoft Defender for Cloud assigns a severity to each alert to help you prioritize which alerts should be investigated first. The severity is based on how confident Microsoft Defender for Cloud is in the finding or the analytic used to issue the alert, as well as the confidence level that there was malicious intent behind the activity that led to the alert.

Additionally, mark resources using tags and create a naming system to identify and categorize Azure resources, especially those processing sensitive data. It is your responsibility to prioritize the remediation of alerts based on the criticality of the Azure resources and environment where the incident occurred.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

IR-6: Containment, eradication and recovery – automate the incident handling

Guidance: Automate manual repetitive tasks to speed up response time and reduce the burden on analysts. Manual tasks take longer to execute, slowing each incident and reducing how many incidents an analyst can handle. Manual tasks also increase analyst fatigue, which increases the risk of human error that causes delays, and degrades the ability of analysts to focus effectively on complex tasks.

Use workflow automation features in Microsoft Defender for Cloud and Microsoft Sentinel to automatically trigger actions or run a playbook to respond to incoming security alerts. The playbook takes actions, such as sending notifications, disabling accounts, and isolating problematic networks.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Posture and Vulnerability Management

For more information, see the Azure Security Benchmark: Posture and Vulnerability Management.

PV-1: Establish secure configurations for Azure services

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-2: Sustain secure configurations for Azure services

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-3: Establish secure configurations for compute resources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-4: Sustain secure configurations for compute resources

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-5: Securely store custom operating system and container images

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-6: Perform software vulnerability assessments

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-7: Rapidly and automatically remediate software vulnerabilities

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

PV-8: Conduct regular attack simulation

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Endpoint Security

For more information, see the Azure Security Benchmark: Endpoint Security.

ES-1: Use Endpoint Detection and Response (EDR)

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

ES-2: Use centrally managed modern anti-malware software

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

ES-3: Ensure anti-malware software and signatures are updated

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Backup and Recovery

For more information, see the Azure Security Benchmark: Backup and Recovery.

BR-1: Ensure regular automated backups

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

BR-2: Encrypt backup data

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

BR-3: Validate all backups including customer-managed keys

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

BR-4: Mitigate risk of lost keys

Guidance: None.

Responsibility: Unset. Please provide a value in the work item.

Microsoft Defender for Cloud monitoring: None

Governance and Strategy

For more information, see the Azure Security Benchmark: Governance and Strategy.

GS-1: Define asset management and data protection strategy

Guidance: Ensure you document and communicate a clear strategy for continuous monitoring and protection of systems and data. Prioritize discovery, assessment, protection, and monitoring of business-critical data and systems.

This strategy should include documented guidance, policy, and standards for the following elements:

  • Data classification standard in accordance with the business risks

  • Security organization visibility into risks and asset inventory

  • Security organization approval of Azure services for use

  • Security of assets through their lifecycle

  • Required access control strategy in accordance with organizational data classification

  • Use of Azure native and third party data protection capabilities

  • Data encryption requirements for in-transit and at-rest use cases

  • Appropriate cryptographic standards

For more information, see the following references:

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-2: Define enterprise segmentation strategy

Guidance: Establish an enterprise-wide strategy to segmenting access to assets using a combination of identity, network, application, subscription, management group, and other controls.

Carefully balance the need for security separation with the need to enable daily operation of the systems that need to communicate with each other and access data.

Ensure that the segmentation strategy is implemented consistently across control types including network security, identity and access models, and application permission/access models, and human process controls.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-3: Define security posture management strategy

Guidance: Continuously measure and mitigate risks to your individual assets and the environment they are hosted in. Prioritize high value assets and highly-exposed attack surfaces, such as published applications, network ingress and egress points, user and administrator endpoints, etc.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-4: Align organization roles, responsibilities, and accountabilities

Guidance: Ensure you document and communicate a clear strategy for roles and responsibilities in your security organization. Prioritize providing clear accountability for security decisions, educating everyone on the shared responsibility model, and educate technical teams on technology to secure the cloud.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-5: Define network security strategy

Guidance: Establish an Azure network security approach as part of your organization’s overall security access control strategy.

This strategy should include documented guidance, policy, and standards for the following elements:

  • Centralized network management and security responsibility

  • Virtual network segmentation model aligned with the enterprise segmentation strategy

  • Remediation strategy in different threat and attack scenarios

  • Internet edge and ingress and egress strategy

  • Hybrid cloud and on-premises interconnectivity strategy

  • Up-to-date network security artifacts (e.g. network diagrams, reference network architecture)

For more information, see the following references:

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-6: Define identity and privileged access strategy

Guidance: Establish an Azure identity and privileged access approaches as part of your organization’s overall security access control strategy.

This strategy should include documented guidance, policy, and standards for the following elements:

  • A centralized identity and authentication system and its interconnectivity with other internal and external identity systems

  • Strong authentication methods in different use cases and conditions

  • Protection of highly privileged users

  • Anomaly user activities monitoring and handling

  • User identity and access review and reconciliation process

For more information, see the following references:

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-7: Define logging and threat response strategy

Guidance: Establish a logging and threat response strategy to rapidly detect and remediate threats while meeting compliance requirements. Prioritize providing analysts with high-quality alerts and seamless experiences so that they can focus on threats rather than integration and manual steps.

This strategy should include documented guidance, policy, and standards for the following elements:

  • The security operations (SecOps) organization’s role and responsibilities

  • A well-defined incident response process aligning with NIST or another industry framework

  • Log capture and retention to support threat detection, incident response, and compliance needs

  • Centralized visibility of and correlation information about threats, using SIEM, native Azure capabilities, and other sources

  • Communication and notification plan with your customers, suppliers, and public parties of interest

  • Use of Azure native and third-party platforms for incident handling, such as logging and threat detection, forensics, and attack remediation and eradication

  • Processes for handling incidents and post-incident activities, such as lessons learned and evidence retention

For more information, see the following references:

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

GS-8: Define backup and recovery strategy

Guidance: Establish an Azure backup and recovery strategy for your organization.

This strategy should include documented guidance, policy, and standards for the following elements:

  • Recovery time objective (RTO) and recovery point objective (RPO) definitions in accordance with your business resiliency objectives

  • Redundancy design in your applications and infrastructure setup

  • Protection of backup using access control and data encryption

For more information, see the following references:

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Next steps