Troubleshoot cluster creation failures with Azure HDInsight
The following issues are most common root causes for cluster creation failures:
- Permission issues
- Resource policy restrictions
- Resource locks
- Unsupported component versions
- Storage account name restrictions
- Service outages
If you are using Azure Data Lake Storage Gen2, and receive the error
Internal server error occurred while processing the request. Please retry the request or contact support., open the Azure portal, go to your Storage account, and under Access Control (IAM), ensure that the Storage Blob Data Contributor or the Storage Blob Data Owner role has Assigned access to the User assigned managed identity for the subscription. See Set up permissions for the managed identity on the Data Lake Storage Gen2 account for detailed instructions.
If you are using Azure Data Lake Storage Gen1, see setup and configuration instructions here. Data Lake Storage Gen1 isn't supported for HBase clusters, and is not supported in HDInsight version 4.0.
If using Azure Storage, ensure that storage account name is valid during the cluster creation.
Resource policy restrictions
Subscription-based Azure policies can deny the creation of public IP addresses. HDInsight cluster creation requires two public IPs.
In general, the following policies can impact cluster creation:
- Policies preventing creation of IP Address & Load balancers within the subscription.
- Policy preventing creation of storage account.
- Policy preventing deletion of networking resources (IP Address /Load Balancers).
Firewalls on your virtual network or storage account can deny communication with HDInsight management IP addresses.
Allow traffic from the IP addresses in the table below.
|Source IP address||Destination||Direction|
Also add the IP addresses specific to the region where the cluster is created. See HDInsight management IP addresses for a listing of the addresses for each Azure region.
If you are using an express route or your own custom DNS server, see Plan a virtual network for Azure HDInsight - connecting multiple networks.
Ensure that there are no locks on your virtual network and resource group. Clusters cannot be created or deleted if the resource group is locked.
Unsupported component versions
Storage account name restrictions
Storage account names cannot be more than 24 characters and cannot contain a special character. These restrictions also apply to the default container name in the storage account.
Other naming restrictions also apply for cluster creation. See Cluster name restrictions, for more information.
Check Azure status for any potential outages or service issues.