Azure security baseline for Data Lake Analytics

  • Artikel
  • 19 minuter för att läsa

This security baseline applies guidance from the Azure Security Benchmark version 1.0 to Data Lake Analytics. The Azure Security Benchmark provides recommendations on how you can secure your cloud solutions on Azure. The content is grouped by the security controls defined by the Azure Security Benchmark and the related guidance applicable to Data Lake Analytics.

Note

Controls not applicable to Data Lake Analytics, or for which the responsibility is Microsoft's, have been excluded. To see how Data Lake Analytics completely maps to the Azure Security Benchmark, see the full Data Lake Analytics security baseline mapping file.

Network Security

For more information, see the Azure Security Benchmark: Network Security.

1.1: Protect Azure resources within virtual networks

Guidance: Use firewall settings for Data Lake Analytics to limit external IP ranges to allow access from your on-premise clients and 3rd party services. Configuration of firewall settings is available via Portal, REST APIs or PowerShell.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

1.4: Deny communications with known-malicious IP addresses

Guidance: Use firewall settings for Data Lake Analytics to limit external IP ranges to allow access from your on-premise clients and 3rd party services. Configuration of firewall settings is available via Portal, REST APIs or PowerShell.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Logging and Monitoring

For more information, see the Azure Security Benchmark: Logging and Monitoring.

2.2: Configure central security log management

Guidance: Ingest logs via Azure Monitor to aggregate security data like Data Lake Analytics 'audit' and 'requests' diagnostics. Within Azure Monitor, use a Log Analytics Workspace to query and perform analytics, and use Azure Storage Accounts for long-term/archival storage, optionally with security features such as immutable storage and enforced retention holds.

Alternatively, you can enable and on-board data to Microsoft Sentinel or a third-party system information and event management solution.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

2.3: Enable audit logging for Azure resources

Guidance: Enable Diagnostic Settings for Data Lake Analytics to access audit and requests logs. These include data like event source, date, user, timestamp and other useful elements.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: The Azure Security Benchmark is the default policy initiative for Microsoft Defender for Cloud and is the foundation for Microsoft Defender for Cloud's recommendations. The Azure Policy definitions related to this control are enabled automatically by Microsoft Defender for Cloud. Alerts related to this control may require a Microsoft Defender plan for the related services.

Azure Policy built-in definitions - Microsoft.DataLakeAnalytics:

Name
(Azure portal)		 Description Effect(s) Version
(GitHub)
Resource logs in Data Lake Analytics should be enabled Audit enabling of resource logs. This enables you to recreate activity trails to use for investigation purposes; when a security incident occurs or when your network is compromised AuditIfNotExists, Disabled 5.0.0

2.5: Configure security log storage retention

Guidance: In Azure Monitor, set your Log Analytics workspace retention period according to your organization's compliance regulations. Use Azure Storage accounts for long-term and archival storage.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

2.6: Monitor and review logs

Guidance: Analyze and monitor logs for anomalous behavior and regularly review results for your Data Lake Analytics resources. Use Azure Monitor's Log Analytics Workspace to review logs and perform queries on log data. Alternatively, you may enable and on-board data to Microsoft Sentinel or a third-party system information and event management solution.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

2.7: Enable alerts for anomalous activities

Guidance: Enable Diagnostic Settings for Data Lake Analytics and send logs to a Log Analytics Workspace. Onboard your Log Analytics Workspace to Microsoft Sentinel as it provides a security orchestration automated response (SOAR) solution. This allows for playbooks (automated solutions) to be created and used to remediate security issues.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Identity and Access Control

For more information, see the Azure Security Benchmark: Identity and Access Control.

3.1: Maintain an inventory of administrative accounts

Guidance: Azure Active Directory (Azure AD) has built-in roles that must be explicitly assigned and are queryable. Use the Azure AD PowerShell module to perform ad hoc queries to discover accounts that are members of administrative groups.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.2: Change default passwords where applicable

Guidance: Data Lake Analytics does not have the concept of default passwords as authentication is provided with Azure Active Directory (Azure AD) and secured by Azure role-based access control (Azure RBAC).

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.3: Use dedicated administrative accounts

Guidance: Create standard operating procedures around the use of dedicated administrative accounts.

You can also enable a Just-In-Time access by using Azure Active Directory (Azure AD) Privileged Identity Management and Azure Resource Manager.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.4: Use Azure Active Directory single sign-on (SSO)

Guidance: Wherever possible, use Azure Active Directory (Azure AD) SSO instead of configuring individual stand-alone credentials per-service. Use Microsoft Defender for Cloud identity and access recommendations.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.5: Use multi-factor authentication for all Azure Active Directory-based access

Guidance: Enable Azure Active Directory (Azure AD) multifactor authentication and follow Microsoft Defender for Cloud Identity and access management recommendations to help protect your Data Lake Analytics resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.6: Use dedicated machines (Privileged Access Workstations) for all administrative tasks

Guidance: Use a secure, Azure-managed workstation (also known as a Privileged Access Workstation, or PAW) for administrative tasks that require elevated privileges.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.7: Log and alert on suspicious activities from administrative accounts

Guidance: Use Azure Active Directory (Azure AD) security reports for generation of logs and alerts when suspicious or unsafe activity occurs in the environment. Use Microsoft Defender for Cloud to monitor identity and access activity.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.8: Manage Azure resources from only approved locations

Guidance: Use Azure Active Directory (Azure AD) named locations to allow access only from specific logical groupings of IP address ranges or countries/regions.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.9: Use Azure Active Directory

Guidance: Use Azure Active Directory (Azure AD) as the central authentication and authorization system. Azure role-based access control (Azure RBAC) provides fine-grained control over a client's access to Data Lake Analytics resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.10: Regularly review and reconcile user access

Guidance: Azure Active Directory (Azure AD) provides logs to help discover stale accounts. In addition, use Azure AD identity and access reviews to efficiently manage group memberships, access to enterprise applications, and role assignments. User access can be reviewed on a regular basis to make sure only the right users have continued access.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.11: Monitor attempts to access deactivated credentials

Guidance: Enable diagnostic settings for Data Lake Analytics and Azure Active Directory (Azure AD), sending all logs to a Log Analytics workspace. Configure desired alerts (such as attempts to access disabled secrets) within Log Analytics.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

3.12: Alert on account sign-in behavior deviation

Guidance: Use Azure Active Directory (Azure AD)'s Risk and Identity Protection features to configure automated responses to detected suspicious actions related to your Data Lake Analytics resources. You should enable automated responses through Microsoft Sentinel to implement your organization's security responses.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Data Protection

For more information, see the Azure Security Benchmark: Data Protection.

4.1: Maintain an inventory of sensitive Information

Guidance: Use tags to assist in tracking Data Lake Analytics resources that store or process sensitive information.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

4.2: Isolate systems storing or processing sensitive information

Guidance: Implement isolation using separate subscriptions, management groups for individual security domains such as environment, data sensitivity. You can restrict your Data Lake Analytics to control the level of access to your Data Lake Analytics resources that your applications and enterprise environments demand. When firewall rules are configured, only applications requesting data over the specified set of networks can access your Data Lake Analytics resources. You can control access to Azure Data Lake Analytics via Azure role-based access control (Azure RBAC).

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

4.3: Monitor and block unauthorized transfer of sensitive information

Guidance: Data loss prevention features are not yet available for Azure Data Lake Analytics resources. Implement third-party solution if required for compliance purposes.

For the underlying platform which is managed by Microsoft, Microsoft treats all customer content as sensitive and guards against customer data loss and exposure. To ensure customer data within Azure remains secure, Microsoft has implemented and maintains a suite of robust data protection controls and capabilities.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

4.4: Encrypt all sensitive information in transit

Guidance: Microsoft Azure resources will negotiate TLS 1.2 by default. Ensure that any clients connecting to your Data Lake Analytics can negotiate using TLS 1.2 or greater.

Responsibility: Shared

Microsoft Defender for Cloud monitoring: None

4.5: Use an active discovery tool to identify sensitive data

Guidance: Data identification features are not yet available for Azure Data Lake Analytics resources. Implement third-party solution if required for compliance purposes.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

4.6: Use Azure RBAC to control access to resources

Guidance: Use Azure role-based access control (Azure RBAC) to control how users interact with the service.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

4.8: Encrypt sensitive information at rest

Guidance: Data is stored in the default Data Lake Storage Gen1 account. For data at rest, Data Lake Storage Gen1 supports "on by default," transparent encryption.

Responsibility: Shared

Microsoft Defender for Cloud monitoring: None

4.9: Log and alert on changes to critical Azure resources

Guidance: Use Azure Monitor with the Azure Activity log to create alerts for when changes take place to production instances of Azure Data Lake Analytics resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Vulnerability Management

For more information, see the Azure Security Benchmark: Vulnerability Management.

5.1: Run automated vulnerability scanning tools

Guidance: Follow recommendations from Microsoft Defender for Cloud on securing your Azure Data Lake Analytics resources.

Microsoft performs vulnerability management on the underlying systems that support Azure Data Lake Analytics.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

5.5: Use a risk-rating process to prioritize the remediation of discovered vulnerabilities

Guidance: Use a common risk scoring program (for example, Common Vulnerability Scoring System) or the default risk ratings provided by your third-party scanning tool.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Inventory and Asset Management

For more information, see the Azure Security Benchmark: Inventory and Asset Management.

6.1: Use automated asset discovery solution

Guidance: Use Azure Resource Graph to query for and discover all resources (such as compute, storage, network, ports, and protocols and so on) in your subscriptions. Ensure appropriate (read) permissions in your tenant and enumerate all Azure subscriptions as well as resources in your subscriptions.

Although classic Azure resources may be discovered via Azure Resource Graph Explorer, it is highly recommended to create and use Azure Resource Manager resources going forward.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

6.2: Maintain asset metadata

Guidance: Apply tags to Azure resources giving metadata to logically organize them into a taxonomy.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

6.3: Delete unauthorized Azure resources

Guidance: Use tagging, management groups, and separate subscriptions, where appropriate, to organize and track Azure Data Lake Analytics resources. Reconcile inventory on a regular basis and ensure unauthorized resources are deleted from the subscription in a timely manner.

In addition, use Azure Policy to put restrictions on the type of resources that can be created in customer subscriptions using the following built-in policy definitions:

  • Not allowed resource types

  • Allowed resource types

Additional information is available at the referenced links.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

6.5: Monitor for unapproved Azure resources

Guidance: Use Azure Policy to put restrictions on the type of resources that can be created in customer subscriptions using the following built-in policy definitions:

  • Not allowed resource types

  • Allowed resource types

In addition, use Azure Resource Graph to query/discover resources within the subscriptions.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

6.9: Use only approved Azure services

Guidance: Use Azure Policy to put restrictions on the type of resources that can be created in customer subscription(s) using the following built-in policy definitions:

  • Not allowed resource types
  • Allowed resource types

Additional information is available at the referenced links

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

6.11: Limit users' ability to interact with Azure Resource Manager

Guidance: Configure Azure Conditional Access to limit users' ability to interact with Azure Resource Manager by configuring "Block access" for the "Microsoft Azure Management" App.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Secure Configuration

For more information, see the Azure Security Benchmark: Secure Configuration.

7.1: Establish secure configurations for all Azure resources

Guidance: Use Azure Policy aliases in the "Microsoft.DataLakeAnalytics" namespace to create custom policies to audit or enforce the configuration of your Azure Data Lake Analytics service. You may also make use of built-in policy definitions related to your Azure Data Lake Analytics, such as:

  • Diagnostic logs in Data Lake Analytics should be enabled

Additional information is available at the referenced links

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

7.3: Maintain secure Azure resource configurations

Guidance: Use Azure Policy [deny] and [deploy if not exist] effects to enforce secure settings across your Azure resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

7.5: Securely store configuration of Azure resources

Guidance: Use Azure Repos to securely store and manage your code like custom Azure policies, Azure Resource Manager templates, Desired State Configuration scripts etc. To access the resources you manage in Azure DevOps, you can grant or deny permissions to specific users, built-in security groups, or groups defined in Azure Active Directory (Azure AD) if integrated with Azure DevOps, or Azure AD if integrated with TFS.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

7.9: Implement automated configuration monitoring for Azure resources

Guidance: Use Azure Policy aliases in the "Microsoft.DataLakeAnalytics" namespace to create custom policies to alert, audit, and enforce system configurations. Use Azure Policy [audit], [deny], and [deploy if not exist] effects to automatically enforce configurations for your Azure Data Lake Analytics resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

7.13: Eliminate unintended credential exposure

Guidance: Implement Credential Scanner to identify credentials within code. Credential Scanner will also encourage moving discovered credentials to more secure locations such as Azure Key Vault.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Malware Defense

For more information, see the Azure Security Benchmark: Malware Defense.

8.2: Pre-scan files to be uploaded to non-compute Azure resources

Guidance: Microsoft anti-malware is enabled on the underlying host that supports Azure services (for example, Azure Data Lake Analytics), however it does not run on customer content.

Pre-scan any content being uploaded to Azure resources, such as App Service, Data Lake Analytics, Blob Storage etc. Microsoft cannot access your data in these instances.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Data Recovery

For more information, see the Azure Security Benchmark: Data Recovery.

9.1: Ensure regular automated back-ups

Guidance: Data Lake Analytics jobs logs and data output are stored in the underlying Data Lake Storage Gen1 service. You can use various methods to copy data including ADLCopy, Azure PowerShell or Azure Data Factory. You can also use Azure Automation to back up data on a regular basis automatically.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

9.2: Perform complete system backups and backup any customer-managed keys

Guidance: Data Lake Analytics jobs logs and data output are stored in the underlying Data Lake Storage Gen1 service. You can use various methods to copy data including ADLCopy, Azure PowerShell or Azure Data Factory.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

9.3: Validate all backups including customer-managed keys

Guidance: Periodically perform data restoration of your backup data to test the integrity of the data.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

9.4: Ensure protection of backups and customer-managed keys

Guidance: Data Lake Analytics backups stored within your Data Lake Storage Gen1 or Azure Storage support encryption by default and cannot be turned off. You should treat your backups as sensitive data and apply the relevant access and data protection controls as part of this baseline.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Incident Response

For more information, see the Azure Security Benchmark: Incident Response.

10.1: Create an incident response guide

Guidance: Build out an incident response guide for your organization. Ensure that there are written incident response plans that define all roles of personnel as well as phases of incident handling/management from detection to post-incident review.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

10.2: Create an incident scoring and prioritization procedure

Guidance: Microsoft Defender for Cloud assigns a severity to each alert to help you prioritize which alerts should be investigated first. The severity is based on how confident Microsoft Defender for Cloud is in the finding or the analytics used to issue the alert as well as the confidence level that there was malicious intent behind the activity that led to the alert.

Additionally, clearly mark subscriptions (for ex. production, non-prod) using tags and create a naming system to clearly identify and categorize Azure resources, especially those processing sensitive data. It is your responsibility to prioritize the remediation of alerts based on the criticality of the Azure resources and environment where the incident occurred.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

10.3: Test security response procedures

Guidance: Conduct exercises to test your systems' incident response capabilities on a regular cadence to help protect your Azure resources. Identify weak points and gaps and then revise your response plan as needed.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

10.4: Provide security incident contact details and configure alert notifications for security incidents

Guidance: Security incident contact information will be used by Microsoft to contact you if the Microsoft Security Response Center (MSRC) discovers that your data has been accessed by an unlawful or unauthorized party. Review incidents after the fact to ensure that issues are resolved.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

10.5: Incorporate security alerts into your incident response system

Guidance: Export your Microsoft Defender for Cloud alerts and recommendations using the Continuous Export feature to help identify risks to Azure resources. Continuous Export allows you to export alerts and recommendations either manually or in an ongoing, continuous fashion. You may use the Microsoft Defender for Cloud data connector to stream the alerts to Microsoft Sentinel.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

10.6: Automate the response to security alerts

Guidance: Use the Workflow Automation feature in Microsoft Defender for Cloud to automatically trigger responses via "Logic Apps" on security alerts and recommendations to protect your Azure resources.

Responsibility: Customer

Microsoft Defender for Cloud monitoring: None

Penetration Tests and Red Team Exercises

For more information, see the Azure Security Benchmark: Penetration Tests and Red Team Exercises.

11.1: Conduct regular penetration testing of your Azure resources and ensure remediation of all critical security findings

Guidance: Follow the Microsoft Cloud Penetration Testing Rules of Engagement to ensure your penetration tests are not in violation of Microsoft policies. Use Microsoft's strategy and execution of Red Teaming and live site penetration testing against Microsoft-managed cloud infrastructure, services, and applications.

Responsibility: Shared

Microsoft Defender for Cloud monitoring: None

Next steps