CheckpointStore Class

CheckpointStore deals with the interaction with the chosen storage service.

It can list and claim partition ownerships as well as list and save checkpoints.

Inheritance

builtins.object

CheckpointStore

Constructor

CheckpointStore()

Methods

claim_ownership	Tries to claim ownership for a list of specified partitions.
list_checkpoints	List the updated checkpoints from the chosen storage service.
list_ownership	Retrieves a complete ownership list from the chosen storage service.
update_checkpoint	Updates the checkpoint using the given information for the offset, associated partition and consumer group in the chosen storage service. Note: If you plan to implement a custom checkpoint store with the intention of running between cross-language EventHubs SDKs, it is recommended to persist the offset value as an integer.

claim_ownership

Tries to claim ownership for a list of specified partitions.

abstract claim_ownership(ownership_list: Iterable[Dict[str, Any]], **kwargs: Any) -> Iterable[Dict[str, Any]]

Parameters

ownership_list

<xref:Iterable>[<xref:Dict>[<xref:str,Any>]]

Required

Iterable of dictionaries containing all the ownerships to claim.

Return type

<xref:Iterable>[<xref:Dict>[<xref:str,Any>]], <xref:Iterable of dictionaries containing partition ownership information:* *fully_qualified_namespace* >(str)<xref:: The fully qualified namespace that the Event Hub belongs to. The format is like "<namespace.servicebus.windows.net". * *eventhub_name* >(str)<xref:: The name of the specific Event Hub the checkpoint is associated with>, <xref:relative to the Event Hubs namespace that contains it. * *consumer_group* >(str)<xref:: The name of the consumer group the ownership are associated with. * *partition_id* >(str)<xref:: The partition ID which the checkpoint is created for. * *owner_id* >(str)<xref:: A UUID representing the owner attempting to claim this partition. * *last_modified_time* >(<xref:UTC datetime.datetime>)<xref:: The last time this ownership was claimed. * *etag* >(str)<xref:: The Etag value for the last time this ownership was modified. Optional depending on storage implementation.>

list_checkpoints

List the updated checkpoints from the chosen storage service.

abstract list_checkpoints(fully_qualified_namespace: str, eventhub_name: str, consumer_group: str, **kwargs: Any) -> Iterable[Dict[str, Any]]

Parameters

fully_qualified_namespace

str

Required

The fully qualified namespace that the Event Hub belongs to. The format is like ".servicebus.windows.net".

eventhub_name

str

Required

The name of the specific Event Hub the checkpoints are associated with, relative to the Event Hubs namespace that contains it.

consumer_group

str

Required

The name of the consumer group the checkpoints are associated with.

Return type

<xref:Iterable>[<xref:Dict>[<xref:str,Any>]], <xref:Iterable of dictionaries containing partition checkpoint information:* *fully_qualified_namespace* >(str)<xref:: The fully qualified namespace that the Event Hub belongs to. The format is like "<namespace.servicebus.windows.net". * *eventhub_name* >(str)<xref:: The name of the specific Event Hub the checkpoints are associated with>, <xref:relative to the Event Hubs namespace that contains it. * *consumer_group* >(str)<xref:: The name of the consumer group the checkpoints are associated with. * *partition_id* >(str)<xref:: The partition ID which the checkpoint is created for. * *sequence_number* >(int)<xref:: The sequence number of the azure.eventhub.EventData. * *offset* >(str)<xref:: The offset of the azure.eventhub.EventData.>

list_ownership

Retrieves a complete ownership list from the chosen storage service.

abstract list_ownership(fully_qualified_namespace: str, eventhub_name: str, consumer_group: str, **kwargs: Any) -> Iterable[Dict[str, Any]]

Parameters

fully_qualified_namespace

str

Required

The fully qualified namespace that the Event Hub belongs to. The format is like ".servicebus.windows.net".

eventhub_name

str

Required

The name of the specific Event Hub the partition ownerships are associated with, relative to the Event Hubs namespace that contains it.

consumer_group

str

Required

The name of the consumer group the ownerships are associated with.

Return type

<xref:Iterable>[<xref:Dict>[str, <xref:Any>]], <xref:Iterable of dictionaries containing partition ownership information:* *fully_qualified_namespace* >(str)<xref:: The fully qualified namespace that the Event Hub belongs to. The format is like "<namespace.servicebus.windows.net". * *eventhub_name* >(str)<xref:: The name of the specific Event Hub the checkpoint is associated with>, <xref:relative to the Event Hubs namespace that contains it. * *consumer_group* >(str)<xref:: The name of the consumer group the ownership are associated with. * *partition_id* >(str)<xref:: The partition ID which the checkpoint is created for. * *owner_id* >(str)<xref:: A UUID representing the current owner of this partition. * *last_modified_time* >(<xref:UTC datetime.datetime>)<xref:: The last time this ownership was claimed. * *etag* >(str)<xref:: The Etag value for the last time this ownership was modified. Optional depending on storage implementation.>

update_checkpoint

Updates the checkpoint using the given information for the offset, associated partition and consumer group in the chosen storage service.

Note: If you plan to implement a custom checkpoint store with the intention of running between cross-language EventHubs SDKs, it is recommended to persist the offset value as an integer.

abstract update_checkpoint(checkpoint: Dict[str, Optional[Union[str, int]]], **kwargs: Any) -> None

Parameters

checkpoint

<xref:Dict>[<xref:str,Any>]

Required

A dict containing checkpoint information:

• fully_qualified_namespace (str): The fully qualified namespace that the Event Hub belongs to. The format is like ".servicebus.windows.net".

• eventhub_name (str): The name of the specific Event Hub the checkpoint is associated with, relative to the Event Hubs namespace that contains it.

• consumer_group (str): The name of the consumer group the checkpoint is associated with.

• partition_id (str): The partition ID which the checkpoint is created for.

• sequence_number (int): The sequence number of the EventData the new checkpoint will be associated with.

• offset (str): The offset of the EventData the new checkpoint will be associated with.

Return type

None