Edit

Share via

Azure HDInsight release notes

  • Article

This article provides information about the most recent Azure HDInsight release updates. For information on earlier releases, see HDInsight Release Notes Archive.

Summary

Azure HDInsight is one of the most popular services among enterprise customers for open-source analytics on Azure. Subscribe to the HDInsight Release Notes for up-to-date information on HDInsight and all HDInsight versions.

To subscribe, click the watch button in the banner and watch out for HDInsight Releases.

Release Information

Release date: May 16, 2024

This release note applies to

HDInsight 5.0 version.

HDInsight 4.0 version.

HDInsight release will be available to all regions over several days. This release note is applicable for image number 2405081840. How to check the image number?

HDInsight uses safe deployment practices, which involve gradual region deployment. It might take up to 10 business days for a new release or a new version to be available in all regions.

OS versions

  • HDInsight 5.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4
  • HDInsight 4.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4

Note

Ubuntu 18.04 is supported under Extended Security Maintenance(ESM) by the Azure Linux team for Azure HDInsight July 2023, release onwards.

For workload specific versions, see HDInsight 5.x component versions.

Fixed issues

  • Added API in gateway to get token for Keyvault, as part of the SFI initiative.
  • In the new Log monitor HDInsightSparkLogs table, for log type SparkDriverLog, some of the fields were missing. For example, LogLevel & Message. This release adds the missing fields to schemas and fixed formatting for SparkDriverLog.
  • Livy logs not available in Log Analytics monitoring SparkDriverLog table, which was due to an issue with Livy log source path and log parsing regex in SparkLivyLog configs.
  • Any HDInsight cluster, using ADLS Gen2 as a primary storage account can leverage MSI based access to any of the Azure resources (for example, SQL, Keyvaults) which is used within the application code.

Coming soon

  • Basic and Standard A-series VMs Retirement.
    • On August 31, 2024, we'll retire Basic and Standard A-series VMs. Before that date, you need to migrate your workloads to Av2-series VMs, which provide more memory per vCPU and faster storage on solid-state drives (SSDs).
    • To avoid service disruptions, migrate your workloads from Basic and Standard A-series VMs to Av2-series VMs before August 31, 2024.
  • Retirement Notifications for HDInsight 4.0 and HDInsight 5.0.

If you have any more questions, contact Azure Support.

You can always ask us about HDInsight on Azure HDInsight - Microsoft Q&A.

We're listening: You’re welcome to add more ideas and other topics here and vote for them - HDInsight Ideas and follow us for more updates on AzureHDInsight Community.

Note

We advise customers to use to latest versions of HDInsight Images as they bring in the best of open source updates, Azure updates and security fixes. For more information, see Best practices.

Next steps