Create Apache Hadoop clusters in HDInsight by using Resource Manager templates
In this article, you learn several ways to create Azure HDInsight clusters using Azure Resource Manager templates. For more information, see Deploy an application with Azure Resource Manager template. To learn about other cluster creation tools and features, click the tab selector on the top of this page or see Cluster creation methods.
Billing for HDInsight clusters is prorated per minute, whether you use them or not. Be sure to delete your cluster after you finish using it. See how to delete an HDInsight cluster.
To follow the instructions in this article, you need:
- An Azure subscription.
- Azure PowerShell and/or Azure Classic CLI.
Resource Manager templates
A Resource Manager template makes it easy to create the following resoruces for your application in a single, coordinated operation:
- HDInsight clusters and their dependent resources (such as the default storage account).
- Other resources (such as Azure SQL Database to use Apache Sqoop).
In the template, you define the resources that are needed for the application. You also specify deployment parameters to input values for different environments. The template consists of JSON and expressions that you use to construct values for your deployment.
You can find HDInsight template samples at Azure Quickstart Templates. Use cross-platform Visual Studio Code with the Resource Manager extension or a text editor to save the template into a file on your workstation.
For more information about Resource Manager templates, see the following articles and examples:
- Author Azure Resource Manager templates
- Deploy an application with Azure Resource Manager templates
- Microsoft.HDInsight/clusters template reference
- Azure Quickstart Templates
Resource Manager enables you to export a Resource Manager template from existing resources in your subscription using different tools. You can use that generated template to learn about the template syntax or to automate the redeployment of your solution as needed.
- Azure portal: See Export an Azure Resource Manager template from existing resources.
- Azure PowerShell: See Export Azure Resource Manager templates with PowerShell.
- Azure Classic CLI: See Export Azure Resource Manager templates with Azure Classic CLI.
Deploy using the portal
You can deploy a Resource Manager template using the Azure portal. For more information, see Deploy resources from custom template.
Deploy using PowerShell
You can deploy a Resource Manager template using Azure PowerShell. For more information, see Deploy resources with Resource Manager templates and Azure PowerShell and Deploy private Resource Manager template with SAS token and Azure PowerShell.
Deploy using Azure CLI
You can deploy a Resource Manager template using classic CLI. For more information, see Deploy resources with Resource Manager templates and Azure CLI and Deploy private Resource Manager template with SAS token and Azure CLI.
Deploy using the REST API
You can deploy a Resource Manager template using REST API. For more information, see Deploy resources with Resource Manager templates and Resource Manager REST API.
Deploy with Visual Studio
Use Visual Studio to create a resource group project and deploy it to Azure through the user interface. You select the type of resources to include in your project. Those resources are automatically added to the Resource Manager template. The project also provides a PowerShell script to deploy the template.
For an introduction to using Visual Studio with resource groups, see Creating and deploying Azure resource groups through Visual Studio.
If you run into issues with creating HDInsight clusters, see access control requirements.
In this article, you have learned several ways to create an HDInsight cluster. To learn more, see the following articles:
- For more HDInsight related templates, see Azure Quickstart Templates.
- For an example of deploying resources through the .NET client library, see Deploy resources by using .NET libraries and a template.
- For an in-depth example of deploying an application, see Provision and deploy microservices predictably in Azure.
- For guidance on deploying your solution to different environments, see Development and test environments in Microsoft Azure.
- To learn about the sections of the Azure Resource Manager template, see Authoring templates.
- For a list of the functions you can use in an Azure Resource Manager template, see Template functions.
We'd love to hear your thoughts. Choose the type you'd like to provide:
Our feedback system is built on GitHub Issues. Read more on our blog.