azure_data_lake_datastore Module

  • Reference

Contains the base functionality for datastores that save connection information to Azure Data Lake Storage.

Classes

AbstractADLSDatastore

Represents the base class for datastores that save connection information to Azure Data Lake Storage.

You should not work with this class directly. To create a datastore that saves connection information to Azure Data Lake Storage, use one of the register_azure_data_lake* methods of the Datastore class.

Initialize a new Azure Data Lake Datastore.
AzureDataLakeDatastore

Represents a datastore that saves connection information to Azure Data Lake Storage.

To create a datastore that saves connection information to Azure Data Lake Storage, use the register_azure_data_lake method of the Datastore class.

Note: When using a datastore to access data, you must have permission to access the data, which depends on the credentials registered with the datastore.

Initialize a new Azure Data Lake Datastore.
AzureDataLakeGen2Datastore

Represents a datastore that saves connection information to Azure Data Lake Storage Gen2.

To create a datastore that saves connection information to Azure Data Lake Storage, use the register_azure_data_lake_gen2 method of the Datastore class.

To access data from an AzureDataLakeGen2Datastore object, create a Dataset and use one of the methods like from_files for a FileDataset. For more information, see Create Azure Machine Learning datasets.

Also keep in mind:

  • The AzureDataLakeGen2 class does not provide upload method, recommended way to uploading data to AzureDataLakeGen2 datastores is via Dataset upload. More details could be found at : https://docs.microsoft.com/azure/machine-learning/how-to-create-register-datasets

  • When using a datastore to access data, you must have permission to access the data, which depends on the credentials registered with the datastore.

  • When using Service Principal Authentication to access storage via AzureDataLakeGen2, the service principal or app registration must be assigned the specific role-based access control (RBAC) role at minimum of "Storage Blob Data Reader". For more information, see Storage built-in roles.

Initialize a new Azure Data Lake Gen2 Datastore.