AzureStorageCheckpointLeaseManager Class
Manages checkpoints and lease with azure storage blobs. In this implementation, checkpoints are data that's actually in the lease blob, so checkpoint operations turn into lease operations under the covers.
- Inheritance
-
AzureStorageCheckpointLeaseManagerAzureStorageCheckpointLeaseManager
Constructor
AzureStorageCheckpointLeaseManager(storage_account_name=None, storage_account_key=None, lease_container_name='eph-leases', storage_blob_prefix=None, lease_renew_interval=10, lease_duration=30, sas_token=None, endpoint_suffix='core.windows.net', connection_string=None)Parameters
- storage_account_name
- str
The storage account name. This is used to authenticate requests signed with an account key and to construct the storage endpoint. It is required unless a connection string is given.
- storage_account_key
- str
The storage account key. This is used for shared key authentication. If neither account key or sas token is specified, anonymous access will be used.
- lease_container_name
- str
The name of the container that will be used to store leases. If it does not already exist it will be created. Default value is 'eph-leases'.
- lease_renew_interval
- int
The interval in seconds at which EPH will attempt to renew the lease of a particular partition. Default value is 10.
- lease_duration
- int
The duration in seconds of a lease on a partition. Default value is 30.
- sas_token
- str
A shared access signature token to use to authenticate requests instead of the account key. If account key and sas token are both specified, account key will be used to sign. If neither are specified, anonymous access will be used.
- endpoint_suffix
- str
The host base component of the url, minus the account name. Defaults to Azure (core.windows.net). Override this to use a National Cloud.
- connection_string
- str
If specified, this will override all other endpoint parameters. See http://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/ for the connection string format.
- connection_string
Methods
| acquire_lease_async |
Acquire the lease on the desired partition for this EventProcessorHost. Note that it is legal to acquire a lease that is already owned by another host. Lease-stealing is how partitions are redistributed when additional hosts are started. |
| create_checkpoint_if_not_exists_async |
Create the given partition checkpoint if it doesn't exist.Do nothing if it does exist. The offset/sequenceNumber for a freshly-created checkpoint should be set to StartOfStream/0. |
| create_checkpoint_store_if_not_exists_async |
Create the checkpoint store if it doesn't exist. Do nothing if it does exist. |
| create_lease_if_not_exists_async |
Create in the store the lease info for the given partition, if it does not exist. Do nothing if it does exist in the store already. |
| create_lease_store_if_not_exists_async |
Create the lease store if it does not exist, do nothing if it does exist. |
| delete_checkpoint_async |
Delete the stored checkpoint for the given partition. If there is no stored checkpoint for the given partition, that is treated as success. |
| delete_lease_async |
Delete the lease info for the given partition from the store. If there is no stored lease for the given partition, that is treated as success. |
| delete_lease_store_async |
Not used by EventProcessorHost, but a convenient function to have for testing. |
| get_all_leases |
Return the lease info for all partitions. A typical implementation could just call get_lease_async() on all partitions. |
| get_checkpoint_async |
Get the checkpoint data associated with the given partition. Could return null if no checkpoint has been created for that partition. |
| get_lease_async |
Return the lease info for the specified partition. Can return null if no lease has been created in the store for the specified partition. |
| initialize |
The EventProcessorHost can't pass itself to the AzureStorageCheckpointLeaseManager constructor because it is still being constructed. Do other initialization here also because it might throw and hence we don't want it in the constructor. |
| release_lease_async |
Give up a lease currently held by this host. If the lease has been stolen, or expired, releasing it is unnecessary, and will fail if attempted. |
| renew_lease_async |
Renew a lease currently held by this host. If the lease has been stolen, or expired, or released, it is not possible to renew it. You will have to call getLease() and then acquireLease() again. |
| update_checkpoint_async |
Update the checkpoint in the store with the offset/sequenceNumber in the provided checkpoint checkpoint:offset/sequeceNumber to update the store with. |
| update_lease_async |
Update the store with the information in the provided lease. It is necessary to currently hold a lease in order to update it. If the lease has been stolen, or expired, or released, it cannot be updated. Updating should renew the lease before performing the update to avoid lease expiration during the process. |
acquire_lease_async
Acquire the lease on the desired partition for this EventProcessorHost. Note that it is legal to acquire a lease that is already owned by another host. Lease-stealing is how partitions are redistributed when additional hosts are started.
async acquire_lease_async(lease)Parameters
Returns
True if the lease was acquired successfully, False if not.
Return type
create_checkpoint_if_not_exists_async
Create the given partition checkpoint if it doesn't exist.Do nothing if it does exist. The offset/sequenceNumber for a freshly-created checkpoint should be set to StartOfStream/0.
async create_checkpoint_if_not_exists_async(partition_id)Parameters
Returns
The checkpoint for the given partition, whether newly created or already existing.
Return type
create_checkpoint_store_if_not_exists_async
Create the checkpoint store if it doesn't exist. Do nothing if it does exist.
async create_checkpoint_store_if_not_exists_async()Returns
True if the checkpoint store already exists or was created OK, False if there was a failure.
Return type
create_lease_if_not_exists_async
Create in the store the lease info for the given partition, if it does not exist. Do nothing if it does exist in the store already.
async create_lease_if_not_exists_async(partition_id)Parameters
Returns
the existing or newly-created lease info for the partition.
Return type
create_lease_store_if_not_exists_async
Create the lease store if it does not exist, do nothing if it does exist.
async create_lease_store_if_not_exists_async()Returns
True if the lease store already exists or was created successfully, False if not.
Return type
delete_checkpoint_async
Delete the stored checkpoint for the given partition. If there is no stored checkpoint for the given partition, that is treated as success.
async delete_checkpoint_async(partition_id)Parameters
delete_lease_async
Delete the lease info for the given partition from the store. If there is no stored lease for the given partition, that is treated as success.
async delete_lease_async(lease)Parameters
delete_lease_store_async
Not used by EventProcessorHost, but a convenient function to have for testing.
async delete_lease_store_async()Returns
True if the lease store was deleted successfully, False if not.
Return type
get_all_leases
Return the lease info for all partitions. A typical implementation could just call get_lease_async() on all partitions.
async get_all_leases()Returns
A list of lease info.
Return type
get_checkpoint_async
Get the checkpoint data associated with the given partition. Could return null if no checkpoint has been created for that partition.
async get_checkpoint_async(partition_id)Parameters
Returns
Given partition checkpoint info, or None if none has been previously stored.
Return type
get_lease_async
Return the lease info for the specified partition. Can return null if no lease has been created in the store for the specified partition.
async get_lease_async(partition_id)Parameters
Returns
lease info for the partition, or None.
Return type
initialize
The EventProcessorHost can't pass itself to the AzureStorageCheckpointLeaseManager constructor because it is still being constructed. Do other initialization here also because it might throw and hence we don't want it in the constructor.
initialize(host)Parameters
- host
release_lease_async
Give up a lease currently held by this host. If the lease has been stolen, or expired, releasing it is unnecessary, and will fail if attempted.
async release_lease_async(lease)Parameters
Returns
True if the lease was released successfully, False if not.
Return type
renew_lease_async
Renew a lease currently held by this host. If the lease has been stolen, or expired, or released, it is not possible to renew it. You will have to call getLease() and then acquireLease() again.
async renew_lease_async(lease)Parameters
Returns
True if the lease was renewed successfully, False if not.
Return type
update_checkpoint_async
Update the checkpoint in the store with the offset/sequenceNumber in the provided checkpoint checkpoint:offset/sequeceNumber to update the store with.
async update_checkpoint_async(lease, checkpoint)Parameters
update_lease_async
Update the store with the information in the provided lease. It is necessary to currently hold a lease in order to update it. If the lease has been stolen, or expired, or released, it cannot be updated. Updating should renew the lease before performing the update to avoid lease expiration during the process.
async update_lease_async(lease)Parameters
Returns
True if the updated was performed successfully, False if not.
Return type
Feedback
Submit and view feedback for