Azure HDInsight: REST APIs

Azure HDInsight is a managed, full-spectrum, open-source analytics service in the cloud for enterprises. You can use open-source frameworks such as Hadoop, Apache Spark, Apache Hive, LLAP, Apache Kafka, Apache Storm, R, and more. You can use these open-source frameworks to enable a broad range of scenarios such as extract, transform, and load (ETL), data warehousing, machine learning, and IoT.

API querying overview

Use these APIs to create and manage HDInsight resources through Azure Resource Manager. To perform operations on Azure HDInsight resources, you send HTTPS request with a supported method: GET, POST, PUT, or DELETE to an endpoint that targets a specific resource. All task operations conform to the HTTP/1.1 protocol specification and each operation returns an x-ms-request-id header that can be used to obtain information about the request. Ensure that requests made to these resources are secure. For more information, see Authenticating Azure Resource Manager requests.

Common parameters and headers

The following information is common to all tasks related to clusters that you might do:

  • Replace {api-version} with 2021-06-01

  • Replace {subscription-id} with your subscription identifier in the URI.

  • Replace {resource-group-name} with the resource group. For more information, see Using Resource groups to manage your Azure resources.

  • Replace {cluster-name} with your HDInsight cluster name in the URI.

  • Set the Content-Type header to application/json.

  • Set the Authorization header to a JSON Web Token that you obtain from Azure Active Directory. For more information, see Authenticating Azure Resource Manager requests.

REST Operation Groups

Operation Group Description
Application Provides operations to create, remove, and retrieve HDInsight clusters with applications.
Cluster Provides operations to work with HDInsight clusters.

Supported REST API versions

The following table lists the supported REST API versions by the Azure HDInsight service. The version must be specified via the x-ms-version header in every request. If not specified, the service defaults to the latest version 2021-06-01.

Version Change introduced Retirement date
2015-03-01-preview 2024-11-30 planned retirement
2018-06-01-preview 2024-11-30 planned retirement
2021-06-01 New version which adds support for several new features detailed below

Updates in the new 2021-06-01 API version

We have released a new version of the HDInsight REST API 2021-06-01. This new version replaces 2018-06-01-preview version which will retire on 2024-11-30. This new API version adds several new features including availability zones, private link, and private end-point connections. With the latest 2021-06-01 API, please note the following new features and changes.

  1. Adding support for creating clusters with availability zones.
  2. Adding support for creating clusters with private link.
  3. Support getting private link resources of HDInsight clusters.
  4. Support get/approve/reject/delete private endpoint connections.
  5. One property {location}, previously optional, will be required.
  6. Two existing properties will be removed from the Locations API, {vmsizes} and {vmsize_filters}. These properties remain in another part of the api, BillingSpecs. We are moving to a single source (BillingSpecs) for these two properties.
  7. Two existing properties will be renamed, to match camelCase style.
    1. {cores_used} will be renamed to {coresUsed}
    2. {max_cores_allowed} will be renamed to {maxCoresAllowed}
    3. {regionName} has been renamed to {regionName}

See also