您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

HDInsight 中具有 Azure 活动目录域服务的企业安全包配置Enterprise Security Package configurations with Azure Active Directory Domain Services in HDInsight

企业安全包 (ESP) 群集在 Azure HDInsight 群集上提供多用户访问。Enterprise Security Package (ESP) clusters provide multiuser access on Azure HDInsight clusters. 具有 ESP 的 HDInsight 群集连接到域,使域用户能够使用其域凭据对群集进行身份验证和运行大数据作业。HDInsight clusters with ESP are connected to a domain so that domain users can use their domain credentials to authenticate with the clusters and run big data jobs.

在本文中,您将了解如何使用 Azure 活动目录域服务 (Azure AD DS) 使用 ESP 配置 HDInsight 群集。In this article, you learn how to configure an HDInsight cluster with ESP by using Azure Active Directory Domain Services (Azure AD DS).

备注

ESP 通常可用于 HDInsight 3.6 和 4.0 中,适用于这些群集类型:Apache Spark、交互式、Hadoop 和 HBase。ESP is generally available in HDInsight 3.6 and 4.0 for these cluster types: Apache Spark, Interactive, Hadoop, and HBase. Apache Kafka 群集类型的 ESP 处于预览状态,仅提供尽力支持。ESP for the Apache Kafka cluster type is in preview with best-effort support only. 不支持在 ESP GA 日期(2018 年 10 月 1 日之前创建的 ESP 群集)。ESP clusters created before the ESP GA date (October 1, 2018) are not supported.

启用 Azure AD DSEnable Azure AD DS

备注

只有租户管理员才具有启用 Azure AD DS 的权限。Only tenant administrators have the privileges to enable Azure AD DS. 如果群集存储是 Azure 数据存储第 1 代或第 2 代,则仅对需要使用基本 Kerberos 身份验证访问群集的用户禁用 Azure 多重身份验证。If the cluster storage is Azure Data Lake Storage Gen1 or Gen2, you must disable Azure Multi-Factor Authentication only for users who will need to access the cluster by using basic Kerberos authentication.

仅当特定用户访问 HDInsight 群集虚拟网络的 IP 范围时,才能使用受信任的 IP条件访问来禁用多因素身份验证。You can use trusted IPs or Conditional Access to disable Multi-Factor Authentication for specific users only when they're accessing the IP range for the HDInsight cluster's virtual network. 如果使用条件访问,请确保 HDInsight 虚拟网络上的 Active Directory 服务终结点已启用。If you're using Conditional Access, make sure that the Active Directory service endpoint in enabled on the HDInsight virtual network.

如果群集存储是 Azure Blob 存储,则不要禁用多重身份验证。If the cluster storage is Azure Blob storage, do not disable Multi-Factor Authentication.

启用 Azure AD DS 是使用 ESP 创建 HDInsight 群集的先决条件。Enabling Azure AD DS is a prerequisite before you can create an HDInsight cluster with ESP. 有关详细信息,请参阅使用Azure 门户启用 Azure 活动目录域服务For more information, see Enable Azure Active Directory Domain Services by using the Azure portal.

启用 Azure AD DS 后,默认情况下,所有用户和对象都开始从 Azure 活动目录 (Azure AD) 同步到 Azure AD DS。When Azure AD DS is enabled, all users and objects start synchronizing from Azure Active Directory (Azure AD) to Azure AD DS by default. 同步操作的时长取决于 Azure AD 中对象的数目。The length of the sync operation depends on the number of objects in Azure AD. 对于数十万个对象,同步可能需要几天时间。The sync might take a few days for hundreds of thousands of objects.

与 Azure AD DS 一起使用的域名必须为 39 个字符或更少,才能使用 HDInsight。The domain name that you use with Azure AD DS must be 39 characters or fewer, to work with HDInsight.

您可以选择仅同步需要访问 HDInsight 群集的组。You can choose to sync only the groups that need access to the HDInsight clusters. 这种仅同步特定组的选项称为“范围有限的同步”。**This option of syncing only certain groups is called scoped synchronization. 有关说明,请参阅将范围化同步从 Azure AD 配置到托管域For instructions, see Configure scoped synchronization from Azure AD to your managed domain.

启用安全 LDAP 时,将域名放在主题名称和证书中的主题替代名称中。When you're enabling secure LDAP, put the domain name in the subject name and the subject alternative name in the certificate. 例如,如果域名为 contoso100.onmicrosoft.com,请确保证书所有者名称和使用者可选名称中存在完全匹配的名称。For example, if your domain name is contoso100.onmicrosoft.com, make sure that exact name exists in your certificate subject name and subject alternative name. 有关详细信息,请参阅为 Azure AD DS 托管域配置安全 LDAPFor more information, see Configure secure LDAP for an Azure AD DS managed domain.

下面的示例创建一个自签名证书。The following example creates a self-signed certificate. 域名contoso100.onmicrosoft.com在(主题名称)SubjectDnsName(主题替代名称)中。The domain name contoso100.onmicrosoft.com is in both Subject (subject name) and DnsName (subject alternative name).

$lifetime=Get-Date
New-SelfSignedCertificate -Subject contoso100.onmicrosoft.com `
  -NotAfter $lifetime.AddDays(365) -KeyUsage DigitalSignature, KeyEncipherment `
  -Type SSLServerAuthentication -DnsName *.contoso100.onmicrosoft.com, contoso100.onmicrosoft.com

检查 Azure AD DS 运行状况状态Check Azure AD DS health status

通过在 "管理" 类别中选择运行状况,查看 Azure 活动目录域服务的运行状况。View the health status of Azure Active Directory Domain Services by selecting Health in the Manage category. 确保 Azure AD DS 的状态为绿色(正在运行),并且同步已完成。Make sure the status of Azure AD DS is green (running) and the synchronization is complete.

Azure AD DS 运行状况

创建和授权托管标识Create and authorize a managed identity

您可以使用用户分配的托管标识来简化和帮助保护域服务操作。You can use a user-assigned managed identity to simplify and help secure domain services operations. HDInsight 域服务参与者角色分配给托管标识时,它可以读取、创建、修改和删除域服务操作。When you assign the HDInsight Domain Services Contributor role to the managed identity, it can read, create, modify, and delete domain services operations.

HDInsight 企业安全包需要某些域服务操作,如创建 OEM 和服务主体。Certain domain services operations, such as creating OUs and service principals, are needed for HDInsight Enterprise Security Package. 您可以在任何订阅中创建托管标识。You can create managed identities in any subscription. 有关托管标识的详细信息,请参阅Azure 资源的托管标识For more information on managed identities in general, see Managed identities for Azure resources. 有关托管标识在 Azure HDInsight 中的工作方式的详细信息,请参阅Azure HDInsight 中的托管标识For more information on how managed identities work in Azure HDInsight, see Managed identities in Azure HDInsight.

要设置 ESP 群集,请创建用户分配的托管标识(如果尚未创建)。To set up ESP clusters, create a user-assigned managed identity if you don't have one already. 有关说明,请参阅使用 Azure 门户创建、列出、删除或将角色分配给用户分配的托管标识For instructions, see Create, list, delete, or assign a role to a user-assigned managed identity by using the Azure portal.

接下来,将HDInsight 域服务参与者角色分配给 Azure AD DS访问控件中的托管标识。Next, assign the HDInsight Domain Services Contributor role to the managed identity in Access control for Azure AD DS. 您需要 Azure AD DS 管理员权限才能分配此角色。You need Azure AD DS admin privileges to make this role assignment.

Azure Active Directory 域服务访问控制

分配HDInsight 域服务参与者角色可确保此标识具有对 Azure AD DS 域执行域服务操作的适当(代表)访问权限。Assigning the HDInsight Domain Services Contributor role ensures that this identity has proper (on behalf of) access to perform domain services operations on the Azure AD DS domain. 这些操作包括创建和删除 O。These operations include creating and deleting OUs.

创建托管标识并给出正确的角色后,Azure AD DS 管理员可以设置谁可以使用此托管标识。After the managed identity is created and given the correct role, the Azure AD DS admin can set up who can use this managed identity. 首先,管理员在门户中选择托管标识,然后在 "概述"下选择访问控制 (IAM)。First, the admin selects the managed identity in the portal, and then selects Access Control (IAM) under Overview. 然后,在右侧,管理员将托管标识操作员角色分配给希望创建 HDInsight ESP 群集的用户或组。Then, on the right, the admin assigns the Managed Identity Operator role to the users or groups that want to create HDInsight ESP clusters.

例如,Azure AD DS 管理员可以将此角色分配给sjmsi托管标识的市场营销团队组,如下图所示。For example, the Azure AD DS admin can assign this role to the MarketingTeam group for the sjmsi managed identity, as shown in the following image. 此分配可确保组织中的正确人员可以使用托管标识创建 ESP 群集。This assignment ensures that the right people in the organization can use the managed identity to create ESP clusters.

HDInsight 托管标识操作者角色分配

网络注意事项Network considerations

备注

Azure AD DS 必须部署在基于 Azure 资源管理器的虚拟网络中。Azure AD DS must be deployed in an Azure Resource Manager-based virtual network. Azure AD DS 不支持经典虚拟网络。Classic virtual networks are not supported for Azure AD DS. 有关详细信息,请参阅使用Azure 门户启用 Azure 活动目录域服务For more information, see Enable Azure Active Directory Domain Services by using the Azure portal.

启用 Azure AD DS 后,本地域名系统 (DNS) 服务器在活动目录虚拟机 (VM) 上运行。After you enable Azure AD DS, a local Domain Name System (DNS) server runs on the Active Directory virtual machines (VMs). 将 Azure AD DS 虚拟网络配置为使用这些自定义 DNS 服务器。Configure your Azure AD DS virtual network to use these custom DNS servers. 要查找正确的 IP 地址,请选择 "管理类别中的属性",然后查看VIRTUAL 网络 上的 IP 地址To locate the right IP addresses, select Properties in the Manage category and look under IP ADDRESS ON VIRTUAL NETWORK.

查找本地 DNS 服务器的 IP 地址

通过在"设置" 类别中选择DNS 服务器,更改 Azure AD DS 虚拟网络中 DNS 服务器的配置以使用这些自定义 IP。Change the configuration of the DNS servers in the Azure AD DS virtual network to use these custom IPs by selecting DNS servers in the Settings category. 然后选择 "自定义" 选项,在文本框中输入第一个 IP 地址,然后选择 "保存"。Then select the Custom option, enter the first IP address in the text box, and select Save. 使用相同的步骤添加更多 IP 地址。Add more IP addresses by using the same steps.

更新虚拟网络 DNS 配置

将 Azure AD DS 实例和 HDInsight 群集放在同一 Azure 虚拟网络中会更方便。It's easier to place both the Azure AD DS instance and the HDInsight cluster in the same Azure virtual network. 如果计划使用不同的虚拟网络,则必须对这些虚拟网络进行对等,以便域控制器对 HDInsight VM 可见。If you plan to use different virtual networks, you must peer those virtual networks so that the domain controller is visible to HDInsight VMs. 有关详细信息,请参阅虚拟网络对等互连For more information, see Virtual network peering.

对等虚拟网络后,将 HDInsight 虚拟网络配置为使用自定义 DNS 服务器,并将 Azure AD DS 专用 IP 作为 DNS 服务器地址输入。After the virtual networks are peered, configure the HDInsight virtual network to use a custom DNS server and enter the Azure AD DS private IPs as the DNS server addresses. 当两个虚拟网络使用相同的 DNS 服务器时,您的自定义域名将解析为正确的 IP,并且可以从 HDInsight 访问。When both virtual networks use the same DNS servers, your custom domain name will resolve to the right IP and will be reachable from HDInsight. 例如,如果您的域名是contoso.com,则在此步骤之后,ping contoso.com应解析为正确的 Azure AD DS IP。For example, if your domain name is contoso.com, then after this step, ping contoso.com should resolve to the right Azure AD DS IP.

为对等虚拟网络配置自定义 DNS 服务器

如果您在 HDInsight 子网中使用网络安全组 (NSG) 规则,则应允许对入站和出站流量所需的 IP。If you're using network security group (NSG) rules in your HDInsight subnet, you should allow the required IPs for both inbound and outbound traffic.

要测试网络设置是否正确,请将 Windows VM 加入 HDInsight 虚拟网络/子网并 ping 域名。To test if your network is set up correctly, join a Windows VM to the HDInsight virtual network/subnet and ping the domain name. (它应该解析为 IP。运行ldp.exe以访问 Azure AD DS 域。(It should resolve to an IP.) Run ldp.exe to access the Azure AD DS domain. 然后,将此 Windows VM 加入域,以确认客户端和服务器之间所需的所有 RPC 调用是否成功。Then join this Windows VM to the domain to confirm that all the required RPC calls succeed between the client and server.

您还可以使用nslookup确认对存储帐户或任何可能使用的外部数据库(例如,外部 Hive 元存储或 Ranger DB)的网络访问。You can also use nslookup to confirm network access to your storage account or any external database that you might use (for example, external Hive metastore or Ranger DB). 如果 NSG 有助于保护 Azure AD DS,请确保 Azure AD DS 子网的 NSG 规则中允许所有必需的端口Make sure that all of the required ports are allowed in the Azure AD DS subnet's NSG rules, if an NSG helps secure Azure AD DS. 如果此 Windows VM 的域加入成功,则可以继续执行下一步并创建 ESP 群集。If the domain joining of this Windows VM is successful, then you can continue to the next step and create ESP clusters.

使用 ESP 创建 HDInsight 群集Create an HDInsight cluster with ESP

正确设置上述步骤后,下一步是创建启用 ESP 的 HDInsight 群集。After you've set up the previous steps correctly, the next step is to create the HDInsight cluster with ESP enabled. 创建 HDInsight 群集时,您可以在 "安全 + 网络" 选项卡上启用企业安全包。如果希望使用 Azure 资源管理器模板进行部署,请使用门户体验一次,并在 "审核 + 创建"页上下载预填充模板以供将来重用。When you create an HDInsight cluster, you can enable Enterprise Security Package on the Security + networking tab. If you prefer to use an Azure Resource Manager template for deployment, use the portal experience once and download the prefilled template on the Review + create page for future reuse.

您还可以在群集创建期间启用HDInsight ID 代理功能。You can also enable the HDInsight ID Broker feature during cluster creation. ID 代理功能允许您使用多重身份验证登录到 Ambari,并获取所需的 Kerberos 票证,而无需在 Azure AD DS 中进行密码哈希。The ID Broker feature lets you sign in to Ambari by using Multi-Factor Authentication and get the required Kerberos tickets without needing password hashes in Azure AD DS.

备注

ESP 群集名称的前六个字符在环境中必须是唯一的。The first six characters of the ESP cluster names must be unique in your environment. 例如,如果在不同的虚拟网络中有多个 ESP 群集,请选择一个命名约定,以确保群集名称上的前六个字符是唯一的。For example, if you have multiple ESP clusters in different virtual networks, choose a naming convention that ensures the first six characters on the cluster names are unique.

Azure HDInsight 企业安全包的域验证

启用 ESP 后,将自动检测和验证与 Azure AD DS 相关的常见错误配置。After you enable ESP, common misconfigurations related to Azure AD DS are automatically detected and validated. 修复这些错误后,可以继续执行下一步。After you fix these errors, you can continue with the next step.

Azure HDInsight 企业安全包失败域验证

使用 ESP 创建 HDInsight 群集时,必须提供以下参数:When you create an HDInsight cluster with ESP, you must supply the following parameters:

  • 群集管理员用户:从同步的 Azure AD DS 实例中选择群集的管理员。Cluster admin user: Choose an admin for your cluster from your synced Azure AD DS instance. 此域帐户必须已同步并在 Azure AD DS 中可用。This domain account must be already synced and available in Azure AD DS.

  • 群集访问组:要同步并有权访问群集的用户的安全组应在 Azure AD DS 中可用。Cluster access groups: The security groups whose users you want to sync and have access to the cluster should be available in Azure AD DS. 例如,HiveUsers 组。An example is the HiveUsers group. 有关详细信息,请参阅在 Azure Active Directory 中创建组并添加成员For more information, see Create a group and add members in Azure Active Directory.

  • LDAPS URLldaps://contoso.com:636一个示例是 。LDAPS URL: An example is ldaps://contoso.com:636.

创建新群集时,可以从用户分配的托管标识下拉列表选择您创建的托管标识。The managed identity that you created can be chosen from the User-assigned managed identity drop-down list when you're creating a new cluster.

Azure HDInsight ESP 活动目录域服务托管标识..

后续步骤Next steps