Open Source Big Data applications that work with Azure Data Lake Storage Gen1

Note

On Feb 29, 2024 Azure Data Lake Storage Gen1 will be retired. For more information, see the official announcement. If you use Azure Data Lake Storage Gen1, make sure to migrate to Azure Data Lake Storage Gen2 prior to that date. To learn how, see Migrate Azure Data Lake Storage from Gen1 to Gen2

Unless you already have an Azure Data Lake Storage Gen1 account, you cannot create new ones.

This article lists the open source big data applications that work with Azure Data Lake Storage Gen1. For the applications in the table below, only the versions available with the listed distribution are supported. For information on what versions of these applications are available with HDInsight, see HDInsight component versioning.

Open Source Software Distribution
Apache Sqoop HDInsight 3.2, 3.4, 3.5, and 3.6
MapReduce HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Storm HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Hive HDInsight 3.2, 3.4, 3.5, and 3.6
HCatalog HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Mahout HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Pig/Pig Latin HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Oozie HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Zookeeper HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Tez HDInsight 3.2, 3.4, 3.5, and 3.6
Apache Spark HDInsight 3.4, 3.5, and 3.6

See also