DataLakeServiceClient Class

A client to interact with the DataLake Service at the account level.

This client provides operations to retrieve and configure the account properties as well as list, create and delete file systems within the account. For operations relating to a specific file system, directory or file, clients for those entities can also be retrieved using the get_client functions.

Inheritance

azure.storage.filedatalake._shared.base_client.StorageAccountHostsMixin

DataLakeServiceClient

Constructor

DataLakeServiceClient(account_url: str, credential: Optional[Any] = None, **kwargs: Any)

Parameters

account_url

str

Required

The URL to the DataLake storage account. Any other entities included in the URL path (e.g. file system or file) will be discarded. This URL can be optionally authenticated with a SAS token.

credential

default value: None

The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential

• except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.

api_version

str

The Storage API version to use for requests. Default value is the most recent service version that is compatible with the current SDK. Setting to an older version may result in reduced feature compatibility.

Examples

Creating the DataLakeServiceClient from connection string.

   from azure.storage.filedatalake import DataLakeServiceClient
   datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string)

Creating the DataLakeServiceClient with Azure Identity credentials.

   from azure.identity import ClientSecretCredential
   token_credential = ClientSecretCredential(
       self.active_directory_tenant_id,
       self.active_directory_application_id,
       self.active_directory_application_secret,
   )
   datalake_service_client = DataLakeServiceClient("https://{}.dfs.core.windows.net".format(self.account_name),
                                                   credential=token_credential)

Variables

url

str

The full endpoint URL to the datalake service endpoint.

primary_endpoint

str

The full primary endpoint URL.

primary_hostname

str

The hostname of the primary endpoint.

Methods

close	This method is to close the sockets opened by the client. It need not be used when using with a context manager.
create_file_system	Creates a new file system under the specified account. If the file system with the same name already exists, a ResourceExistsError will be raised. This method returns a client with which to interact with the newly created file system.
delete_file_system	Marks the specified file system for deletion. The file system and any files contained within it are later deleted during garbage collection. If the file system is not found, a ResourceNotFoundError will be raised.
from_connection_string	Create DataLakeServiceClient from a Connection String. :return a DataLakeServiceClient :rtype ~azure.storage.filedatalake.DataLakeServiceClient
get_directory_client	Get a client to interact with the specified directory. The directory need not already exist.
get_file_client	Get a client to interact with the specified file. The file need not already exist.
get_file_system_client	Get a client to interact with the specified file system. The file system need not already exist.
get_service_properties	Gets the properties of a storage account's datalake service, including Azure Storage Analytics. New in version 12.4.0: This operation was introduced in API version '2020-06-12'.
get_user_delegation_key	Obtain a user delegation key for the purpose of signing SAS tokens. A token credential must be present on the service object for this request to succeed.
list_file_systems	Returns a generator to list the file systems under the specified account. The generator will lazily follow the continuation tokens returned by the service and stop when all file systems have been returned.
set_service_properties	Sets the properties of a storage account's Datalake service, including Azure Storage Analytics. New in version 12.4.0: This operation was introduced in API version '2020-06-12'. If an element (e.g. analytics_logging) is left as None, the existing settings on the service for that functionality are preserved.
undelete_file_system	Restores soft-deleted filesystem. Operation will only be successful if used within the specified number of days set in the delete retention policy. New in version 12.3.0: This operation was introduced in API version '2019-12-12'.

close

This method is to close the sockets opened by the client. It need not be used when using with a context manager.

close() -> None

create_file_system

Creates a new file system under the specified account.

If the file system with the same name already exists, a ResourceExistsError will be raised. This method returns a client with which to interact with the newly created file system.

create_file_system(file_system: Union[FileSystemProperties, str], metadata: Optional[Dict[str, str]] = None, public_access: Optional[PublicAccess] = None, **kwargs) -> FileSystemClient

Parameters

file_system

str

Required

The name of the file system to create.

metadata

dict(str, str)

default value: None

A dict with name-value pairs to associate with the file system as metadata. Example: {'Category':'test'}

public_access

PublicAccess

default value: None

Possible values include: file system, file.

timeout

int

The timeout parameter is expressed in seconds.

Return type

FileSystemClient

Examples

Creating a file system in the datalake service.

   datalake_service_client.create_file_system("filesystem")

delete_file_system

Marks the specified file system for deletion.

The file system and any files contained within it are later deleted during garbage collection. If the file system is not found, a ResourceNotFoundError will be raised.

delete_file_system(file_system: Union[FileSystemProperties, str], **kwargs) -> FileSystemClient

Parameters

file_system

str or FileSystemProperties

Required

The file system to delete. This can either be the name of the file system, or an instance of FileSystemProperties.

lease

DataLakeLeaseClient or str

If specified, delete_file_system only succeeds if the file system's lease is active and matches this ID. Required if the file system has an active lease.

if_modified_since

datetime

A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.

if_unmodified_since

datetime

A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.

etag

str

An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.

match_condition

MatchConditions

The match condition to use upon the etag.

timeout

int

The timeout parameter is expressed in seconds.

Return type

None

Examples

Deleting a file system in the datalake service.

   datalake_service_client.delete_file_system("filesystem")

from_connection_string

Create DataLakeServiceClient from a Connection String.

:return a DataLakeServiceClient :rtype ~azure.storage.filedatalake.DataLakeServiceClient

from_connection_string(conn_str: str, credential: Optional[Any] = None, **kwargs: Any) -> ClassType

Parameters

conn_str

str

Required

A connection string to an Azure Storage account.

credential

Required

The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an instance of a AzureSasCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string.

credential

default value: None

Examples

Creating the DataLakeServiceClient from a connection string.

   from azure.storage.filedatalake import DataLakeServiceClient
   datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string)

get_directory_client

Get a client to interact with the specified directory.

The directory need not already exist.

get_directory_client(file_system: Union[FileSystemProperties, str], directory: Union[DirectoryProperties, str]) -> DataLakeDirectoryClient

Parameters

file_system

str or FileSystemProperties

Required

The file system that the directory is in. This can either be the name of the file system, or an instance of FileSystemProperties.

directory

str or DirectoryProperties

Required

The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.

Returns

A DataLakeDirectoryClient.

Return type

DataLakeDirectoryClient

Examples

Getting the directory client to interact with a specific directory.

   directory_client = datalake_service_client.get_directory_client(file_system_client.file_system_name,
                                                                   "mydirectory")

get_file_client

Get a client to interact with the specified file.

The file need not already exist.

get_file_client(file_system: Union[FileSystemProperties, str], file_path: Union[FileProperties, str]) -> DataLakeFileClient

Parameters

file_system

str or FileSystemProperties

Required

The file system that the file is in. This can either be the name of the file system, or an instance of FileSystemProperties.

file_path

str or FileProperties

Required

The file with which to interact. This can either be the full path of the file(from the root directory), or an instance of FileProperties. eg. directory/subdirectory/file

Returns

A DataLakeFileClient.

Return type

DataLakeFileClient

Examples

Getting the file client to interact with a specific file.

   file_client = datalake_service_client.get_file_client(file_system_client.file_system_name, "myfile")

get_file_system_client

Get a client to interact with the specified file system.

The file system need not already exist.

get_file_system_client(file_system: Union[FileSystemProperties, str]) -> FileSystemClient

Parameters

file_system

str or FileSystemProperties

Required

The file system. This can either be the name of the file system, or an instance of FileSystemProperties.

Returns

A FileSystemClient.

Return type

FileSystemClient

Examples

Getting the file system client to interact with a specific file system.

   # Instantiate a DataLakeServiceClient using a connection string
   from azure.storage.filedatalake import DataLakeServiceClient
   datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string)

   # Instantiate a FileSystemClient
   file_system_client = datalake_service_client.get_file_system_client("mynewfilesystem")

get_service_properties

Gets the properties of a storage account's datalake service, including Azure Storage Analytics.

New in version 12.4.0: This operation was introduced in API version '2020-06-12'.

get_service_properties(**kwargs: Any) -> Dict[str, Any]

Parameters

timeout

int

The timeout parameter is expressed in seconds.

Returns

An object containing datalake service properties such as analytics logging, hour/minute metrics, cors rules, etc.

Return type

<xref:Dict>[str, <xref:Any>]

get_user_delegation_key

Obtain a user delegation key for the purpose of signing SAS tokens. A token credential must be present on the service object for this request to succeed.

get_user_delegation_key(key_start_time: datetime, key_expiry_time: datetime, **kwargs: Any) -> UserDelegationKey

Parameters

key_start_time

datetime

Required

A DateTime value. Indicates when the key becomes valid.

key_expiry_time

datetime

Required

A DateTime value. Indicates when the key stops being valid.

timeout

int

The timeout parameter is expressed in seconds.

Returns

The user delegation key.

Return type

UserDelegationKey

Examples

Get user delegation key from datalake service client.

   from datetime import datetime, timedelta
   user_delegation_key = datalake_service_client.get_user_delegation_key(datetime.utcnow(),
                                                                         datetime.utcnow() + timedelta(hours=1))

list_file_systems

Returns a generator to list the file systems under the specified account.

The generator will lazily follow the continuation tokens returned by the service and stop when all file systems have been returned.

list_file_systems(name_starts_with: Optional[str] = None, include_metadata: Optional[bool] = None, **kwargs) -> ItemPaged[FileSystemProperties]

Parameters

name_starts_with

str

default value: None

Filters the results to return only file systems whose names begin with the specified prefix.

include_metadata

bool

default value: None

Specifies that file system metadata be returned in the response. The default value is False.

results_per_page

int

The maximum number of file system names to retrieve per API call. If the request does not specify the server will return up to 5,000 items per page.

timeout

int

The timeout parameter is expressed in seconds.

include_deleted

bool

Specifies that deleted file systems to be returned in the response. This is for file system restore enabled account. The default value is False. .. versionadded:: 12.3.0

include_system

bool

Flag specifying that system filesystems should be included. .. versionadded:: 12.6.0

Returns

An iterable (auto-paging) of FileSystemProperties.

Return type

ItemPaged[FileSystemProperties]

Examples

Listing the file systems in the datalake service.

   file_systems = datalake_service_client.list_file_systems()
   for file_system in file_systems:
       print(file_system.name)

set_service_properties

Sets the properties of a storage account's Datalake service, including Azure Storage Analytics.

New in version 12.4.0: This operation was introduced in API version '2020-06-12'.

If an element (e.g. analytics_logging) is left as None, the existing settings on the service for that functionality are preserved.

set_service_properties(**kwargs: Any) -> None

Parameters

analytics_logging

Groups the Azure Analytics Logging settings.

hour_metrics

The hour metrics settings provide a summary of request statistics grouped by API in hourly aggregates.

minute_metrics

The minute metrics settings provide request statistics for each minute.

cors

You can include up to five CorsRule elements in the list. If an empty list is specified, all CORS rules will be deleted, and CORS will be disabled for the service.

target_version

str

Indicates the default version to use for requests if an incoming request's version is not specified.

delete_retention_policy

The delete retention policy specifies whether to retain deleted files/directories. It also specifies the number of days and versions of file/directory to keep.

static_website

Specifies whether the static website feature is enabled, and if yes, indicates the index document and 404 error document to use.

timeout

int

The timeout parameter is expressed in seconds.

Return type

None

undelete_file_system

Restores soft-deleted filesystem.

Operation will only be successful if used within the specified number of days set in the delete retention policy.

New in version 12.3.0: This operation was introduced in API version '2019-12-12'.

undelete_file_system(name: str, deleted_version: str, **kwargs: Any) -> azure.storage.filedatalake._file_system_client.FileSystemClient

Parameters

name

str

Required

Specifies the name of the deleted filesystem to restore.

deleted_version

str

Required

Specifies the version of the deleted filesystem to restore.

timeout

int

The timeout parameter is expressed in seconds.

Return type

FileSystemClient