Data sharding policy

The sharding policy defines if and how Extents (data shards) in the Azure Data Explorer cluster should be sealed.

Note

The policy applies to all operations that create new extents, such as commands for data ingestion, and .merge and .rebuild commands

The data sharding policy contains the following properties:

  • MaxRowCount:

    • Maximum row count for an extent created by an ingestion or rebuild operation.
    • Defaults to 750,000.
    • Not in effect for merge operations.
      • If you must limit the number of rows in extents created by merge operations, adjust the RowCountUpperBoundForMerge property in the entity's extents merge policy.
  • MaxExtentSizeInMb:

    • Maximum allowed compressed data size (in megabytes) for an extent created by a merge operation.
    • In effect only for merge operations.
    • Defaults to 1,024 (1GB).
  • MaxOriginalSizeInMb:

    • Maximum allowed original data size (in megabytes) for an extent created by a rebuild operation.
    • In effect only for rebuild operations.
    • Defaults to 2,048 (2GB).

Warning

Consult with the Azure Data Explorer team before altering a data sharding policy.

When a database is created, it contains the default data sharding policy. This policy is inherited by all tables created in the database (unless the policy is explicitly overridden at the table level).

Use the sharding policy control commands) to manage data sharding policies for databases and tables.