Edit

ADLS Adapter

  • Article

The ADLS adapter is the storage adapter that's used to interact with data on Azure Data Lake Storage Gen2.

public class ADLSAdapter extends NetworkAdapter

ADLSAdapter extends NetworkAdapter, StorageAdapterBase in Python.

Constructors

Name Description
ADLSAdapter() Initializes a new instance of the ADLSAdapter class. The user must apply an adapter configuration if this constructor is used. See UpdateConfig(...).
ADLSAdapter(string, string, string, string, string, AzureCloudEndpoint)
hostname: The ADLS hostname.
root: The root path of the schema documents.
tenant: The tenant.
clientId: The client ID of the application accessing ADLS.
secret: The secret for the application accessing ADLS.
endpoint [optional]: The national cloud authentication endpoint, default is the public cloud endpoint.		 Initializes a new instance of the ADLSAdapter class with client ID/secret authentication.
ADLSAdapter(string, string, string)
hostname: The ADLS hostname.
root: The root path of the schema documents.
sharedKey: The account/shared key.		 Initializes a new instance of the ADLSAdapter class with shared key authentication.
ADLSAdapter(string, string, TokenProvider)
hostname: The ADLS hostname.
root: The root path of the schema documents.
tokenProvider: The user-defined token provider.		 Initializes a new instance of the ADLSAdapter class with a user-defined implementation of the TokenProvider interface.

Properties

Name Type Description
Root string The root path of the schema documents.
Hostname string The ADLS hostname.
Tenant string The tenant.
ClientId string The client ID of the application accessing ADLS.
Secret string The secret for the application accessing ADLS.
Endpoint AzureCloudEndpoint The Azure AD authentication endpoint for the application accessing ADLS.
SharedKey string The account or shared key.
SasToken string The SAS token.
Token Provider TokenProvider The token provider used to dynamically generate the access token.
LocationHint string The hint given to the reader application about where to find the the adapter implementation (for example, in Nuget, NPM, etc.).

Methods

Name Description Return Type
CanRead() Returns true, since the ADLS adapter can read data. bool
CanWrite() Returns true, since the ADLS adapter can write data to its source. bool
ReadAsync(string) See StorageAdapterBase.ReadAsync(...). Task<string>
WriteAsync(string, string) See StorageAdapterBase.WriteAsync(...). Write operation is a 3-step process:
HTTP PUT: Creates an empty file.
HTTP PATCH: An Append action to upload content to ADLS.
HTTP PATCH: A Flush action to add the content to the file.
A 0B file gets created if steps 2 and/or 3 fail. In such cases, WriteAsync operation deletes the empty file by default. Please set the following feature flag to false in CdmCorpusContext to skip deletion of empty file: ADLSAdapter_deleteEmptyFile.		 Task
CreateAdapterPath(string) See StorageAdapterBase.CreateAdapterPath(...). string
CreateCorpusPath(string) See StorageAdapterBase.CreateCorpusPath(...). string
ClearCache() See StorageAdapterBase.ClearCache(). This method doesn't do anything if the ADLS adapter doesn't maintain a cache. void
ComputeLastModifiedTimeAsync(string) See StorageAdapterBase.ComputeLastModifiedTimeAsync(...). Task<DateTimeOffset?>
FetchAllFilesAsync(string) See StorageAdapterBase.FetchAllFilesAsync(...). Task<List<string>>
FetchAllFilesMetadataAsync(string) See StorageAdapterBase.FetchAllFilesMetadataAsync(...). Task<IDictionary<string, CdmFileMetadata>>
FetchFileMetadataAsync(string) See StorageAdapterBase.FetchFileMetadataAsync(...). CdmFileMetadata
FetchConfig() See StorageAdapterBase.FetchConfig(). string
UpdateConfig(string) See StorageAdapterBase.UpdateConfig(...). void
CreateFileQueryCacheContext() See StorageAdapterBase.CreateFileQueryCacheContext(). IDisposable