您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

在 HDInsight 中的 Apache Hadoop 群集上使用空边缘节点Use empty edge nodes on Apache Hadoop clusters in HDInsight

了解如何将空边缘节点添加到 HDInsight 群集。Learn how to add an empty edge node to an HDInsight cluster. 空边缘节点是安装并配置了与头节点中相同的客户端工具,但未运行 Apache Hadoop 服务的 Linux 虚拟机。An empty edge node is a Linux virtual machine with the same client tools installed and configured as in the headnodes, but with no Apache Hadoop services running. 可以使用该边缘节点来访问群集、测试客户端应用程序和托管客户端应用程序。You can use the edge node for accessing the cluster, testing your client applications, and hosting your client applications.

可以将空边缘节点添加到现有 HDInsight 群集,或者在创建群集时将此类节点添加到新群集。You can add an empty edge node to an existing HDInsight cluster, to a new cluster when you create the cluster. 添加空边缘节点的操作是使用 Azure Resource Manager 模板完成的。Adding an empty edge node is done using Azure Resource Manager template. 以下示例演示如何使用模板执行此操作:The following sample demonstrates how it is done using a template:

"resources": [
    {
        "name": "[concat(parameters('clusterName'),'/', variables('applicationName'))]",
        "type": "Microsoft.HDInsight/clusters/applications",
        "apiVersion": "2015-03-01-preview",
        "dependsOn": [ "[concat('Microsoft.HDInsight/clusters/',parameters('clusterName'))]" ],
        "properties": {
            "marketPlaceIdentifier": "EmptyNode",
            "computeProfile": {
                "roles": [{
                    "name": "edgenode",
                    "targetInstanceCount": 1,
                    "hardwareProfile": {
                        "vmSize": "{}"
                    }
                }]
            },
            "installScriptActions": [{
                "name": "[concat('emptynode','-' ,uniquestring(variables('applicationName')))]",
                "uri": "[parameters('installScriptAction')]",
                "roles": ["edgenode"]
            }],
            "uninstallScriptActions": [],
            "httpsEndpoints": [],
            "applicationType": "CustomApplication"
        }
    }
],

如示例中所示,可以选择性地调用脚本操作来执行其他配置,例如,在边缘节点中安装 Apache HueAs shown in the sample, you can optionally call a script action to perform additional configuration, such as installing Apache Hue in the edge node. 脚本操作脚本必须可在 Web 上公开访问。The script action script must be publicly accessible on the web. 例如,如果该脚本存储在 Azure 存储中,请使用公共容器或公共 blob。For example, if the script is stored in Azure storage, use either public containers or public blobs.

边缘节点虚拟机大小必须满足 HDInsight 群集工作节点 vm 的大小要求。The edge node virtual machine size must meet the HDInsight cluster worker node vm size requirements. 有关建议的工作节点 vm 的大小信息,请参阅在 HDInsight 中创建 Apache Hadoop 群集For the recommended worker node vm sizes, see Create Apache Hadoop clusters in HDInsight.

创建边缘节点后,可以使用 SSH 连接到该节点,运行客户端工具访问 HDInsight 中的 Hadoop 群集。After you have created an edge node, you can connect to the edge node using SSH, and run client tools to access the Hadoop cluster in HDInsight.

警告

安装在空边缘节点上的自定义组件将从 Microsoft 得到商业上合理的支持。Custom components that are installed on the edge node receive commercially reasonable support from Microsoft. 这或许可解决遇到的问题。This might result in resolving problems you encounter. 或者也可参考社区资源,获取更多帮助。Or, you may be referred to community resources for further assistance. 以下是一些可获得社区帮助的最活跃站点:The following are some of the most active sites for getting help from the community:

如果在使用 Apache 技术,可通过 https://apache.org 上的 Apache 项目站点(如 Apache Hadoop 站点)获取帮助。If you are using an Apache technology, you may be able to find assistance through the Apache project sites on https://apache.org, such as the Apache Hadoop site.

重要

Ubuntu 映像可在发布后的 3 个月内用于创建新的 HDInsight 群集。Ubuntu images become available for new HDInsight cluster creation within 3 months of being published. 自 2019 年 1 月起,运行的群集(包括边缘节点)不进行自动修补。As of January 2019, running clusters (including edge nodes) are not auto-patched. 客户必须使用脚本操作或其他机制来修补正在运行的群集。Customers must use script actions or other mechanisms to patch a running cluster. 有关详细信息,请参阅针对 HDInsight 的 OS 修补For more information, see OS patching for HDInsight.

将边缘节点添加到现有群集Add an edge node to an existing cluster

本部分介绍如何使用 Resource Manager 模板将边缘节点添加到现有 HDInsight 群集。In this section, you use a Resource Manager template to add an edge node to an existing HDInsight cluster. 可以在 GitHub 中找到 Resource Manager 模板。The Resource Manager template can be found in GitHub. 资源管理器模板调用位于 https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-hdinsight-linux-add-edge-node/scripts/EmptyNodeSetup.sh的脚本操作。脚本不执行任何操作。The Resource Manager template calls a script action located at https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-hdinsight-linux-add-edge-node/scripts/EmptyNodeSetup.sh. The script doesn't perform any actions. 它只是演示如何从 Resource Manager 模板调用脚本操作。It is to demonstrate calling script action from a Resource Manager template.

将空边缘节点添加到现有群集To add an empty edge node to an existing cluster

  1. 单击以下图像登录到 Azure,然后在 Azure 门户中打开 Azure Resource Manager 模板。Click the following image to sign in to Azure and open the Azure Resource Manager template in the Azure portal.

    Deploy to Azure button for new cluster

  2. 配置以下属性:Configure the following properties:

    • 订阅:选择用于创建群集的 Azure 订阅。Subscription: Select an Azure subscription used for creating the cluster.
    • 资源组:选择现有 HDInsight 群集所用的资源组。Resource group: Select the resource group used for the existing HDInsight cluster.
    • 位置:选择现有 HDInsight 群集的位置。Location: Select the location of the existing HDInsight cluster.
    • 群集名:输入现有 HDInsight 群集的名称。Cluster Name: Enter the name of an existing HDInsight cluster.
    • 边缘节点大小:选择一个 VM 大小。Edge Node Size: Select one of the VM sizes. vm 的大小必须满足工作节点 vm 的大小要求。The vm size must meet the worker node vm size requirements. 有关建议的工作节点 vm 的大小信息,请参阅在 HDInsight 中创建 Apache Hadoop 群集For the recommended worker node vm sizes, see Create Apache Hadoop clusters in HDInsight.
    • 边缘节点前缀:默认值为 newEdge Node Prefix: The default value is new. 如果使用默认值,边缘节点的名称为 new-edgenodeUsing the default value, the edge node name is new-edgenode. 可以通过门户自定义前缀。You can customize the prefix from the portal. 也可以通过模板自定义完整名称。You can also customize the full name from the template.
  3. 选中“我同意上述条款和条件”,并单击“购买”创建边缘节点。Check I agree to the terms and conditions stated above, and then click Purchase to create the edge node.

重要

请确保选择现有 HDInsight 群集的 Azure 资源组。Make sure to select the Azure resource group for the existing HDInsight cluster. 否则,会收到错误消息“无法对嵌套资源执行请求的操作。Otherwise, you get the error message "Can not perform requested operation on nested resource. 父资源 '<ClusterName>' 未找到。”Parent resource '<ClusterName>' not found."

创建群集时添加边缘节点Add an edge node when creating a cluster

本部分介绍如何使用 Resource Manager 模板创建包含边缘节点的 HDInsight 群集。In this section, you use a Resource Manager template to create HDInsight cluster with an edge node. 可以在Azure 快速入门模板库中找到资源管理器模板。The Resource Manager template can be found in the Azure quickstart templates gallery. 资源管理器模板调用位于 https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-hdinsight-linux-with-edge-node/scripts/EmptyNodeSetup.sh的脚本操作。脚本不执行任何操作。The Resource Manager template calls a script action located at https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-hdinsight-linux-with-edge-node/scripts/EmptyNodeSetup.sh. The script doesn't perform any actions. 它只是演示如何从 Resource Manager 模板调用脚本操作。It is to demonstrate calling script action from a Resource Manager template.

创建包含边缘节点的 HDInsight 群集To create an HDInsight cluster with an edge node

  1. 创建一个 HDInsight 群集(如果没有)。Create an HDInsight cluster if you don't have one yet. 请参阅 开始使用 HDInsight 中的 HadoopSee Get started using Hadoop in HDInsight.

  2. 单击以下图像登录到 Azure,然后在 Azure 门户中打开 Azure Resource Manager 模板。Click the following image to sign in to Azure and open the Azure Resource Manager template in the Azure portal.

    Deploy to Azure button for new cluster

  3. 配置以下属性:Configure the following properties:

    • 订阅:选择用于创建群集的 Azure 订阅。Subscription: Select an Azure subscription used for creating the cluster.

    • 资源组:创建用于群集的新资源组。Resource group: Create a new resource group used for the cluster.

    • 位置:选择资源组的位置。Location: Select a location for the resource group.

    • 群集名:输入要创建的新群集的名称。Cluster Name: Enter a name for the new cluster to create.

    • 群集登录用户名:输入 Hadoop HTTP 用户名。Cluster Login User Name: Enter the Hadoop HTTP user name. 默认名称为 adminThe default name is admin.

    • 群集登录密码:输入 Hadoop HTTP 用户密码。Cluster Login Password: Enter the Hadoop HTTP user password.

    • SSH 用户名:输入 SSH 用户名。Ssh User Name: Enter the SSH user name. 默认名称为 sshuserThe default name is sshuser.

    • SSH 密码:输入 SSH 用户密码。Ssh Password: Enter the SSH user password.

    • 安装脚本操作:保留此文章的默认值。Install Script Action: Keep the default value for going through this article.

      模板中已硬编码某些属性:群集类型、群集辅助角色节点计数、边缘节点大小和边缘节点名称。Some properties have been hardcoded in the template: Cluster type, Cluster worker node count, Edge node size, and Edge node name.

  4. 选中“我同意上述条款和条件”,并单击“购买”,以创建包含该边缘节点的群集。Check I agree to the terms and conditions stated above, and then click Purchase to create the cluster with the edge node.

添加多个边缘节点Add multiple edge nodes

可以向一个 HDInsight 群集添加多个边缘节点。You can add multiple edge nodes to an HDInsight cluster. 只能使用 Azure 资源管理器模板进行多边缘节点配置。The multiple edge nodes configuration can only be done using Azure Resource Manager Templates. 请参阅本文开头的模板示例。See the template sample at the beginning of this article. 需更新 targetInstanceCount,使之反映要创建的边缘节点数。You need to update the targetInstanceCount to reflect the number of edge nodes you would like to create.

访问边缘节点Access an edge node

边缘节点 ssh 终结点为<边缘节点名称<群集名称>-ssh.azurehdinsight.cn:22。The edge node ssh endpoint is <EdgeNodeName>.<ClusterName>-ssh.azurehdinsight.net:22. 例如,new-edgenode.myedgenode0914-ssh.azurehdinsight.net:22。For example, new-edgenode.myedgenode0914-ssh.azurehdinsight.net:22.

在 Azure 门户上,边缘节点显示为应用程序。The edge node appears as an application on the Azure portal. 门户中提供了使用 SSH 访问边缘节点时所需的信息。The portal gives you the information to access the edge node using SSH.

验证边缘节点 SSH 终结点To verify the edge node SSH endpoint

  1. 登录到 Azure 门户Sign on to the Azure portal.
  2. 打开包含边缘节点的 HDInsight 群集。Open the HDInsight cluster with an edge node.
  3. 单击“应用程序”。Click Applications. 此时会显示该边缘节点。You shall see the edge node. 默认名称为 new-edgenodeThe default name is new-edgenode.
  4. 单击该边缘节点。Click the edge node. 此时会显示 SSH 终结点。You shall see the SSH endpoint.

在边缘节点上使用 HiveTo use Hive on the edge node

  1. 使用 SSH 连接到边缘节点。Use SSH to connect to the edge node. 有关信息,请参阅将 SSH 与 HDInsight 配合使用For information, see Use SSH with HDInsight.

  2. 使用 SSH 连接到边缘节点后,使用以下命令打开 Hive 控制台:After you have connected to the edge node using SSH, use the following command to open the Hive console:

     hive
    
  3. 运行以下命令显示群集中的 Hive 表:Run the following command to show Hive tables in the cluster:

     show tables;
    

删除边缘节点Delete an edge node

可以在 Azure 门户中删除边缘节点。You can delete an edge node from the Azure portal.

访问边缘节点To access an edge node

  1. 登录到 Azure 门户Sign on to the Azure portal.
  2. 打开包含边缘节点的 HDInsight 群集。Open the HDInsight cluster with an edge node.
  3. 单击“应用程序”。Click Applications. 此时会显示边缘节点的列表。You shall see a list of edge nodes.
  4. 右键单击要删除的边缘节点,并单击“删除”。Right-click the edge node you want to delete, and then click Delete.
  5. 单击“是”确认。Click Yes to confirm.

后续步骤Next steps

本文介绍了如何添加边缘节点以及如何访问边缘节点。In this article, you have learned how to add an edge node and how to access the edge node. 若要了解更多信息,请参阅下列文章:To learn more, see the following articles: