Azure Cognitive Search monitoring data reference

This article provides a reference of log and metric data collected to analyze the performance and availability of Azure Cognitive Search. See Monitoring Azure Cognitive Search for details on collecting and analyzing monitoring data for your search service.

Metrics

This section lists the platform metrics collected for Azure Cognitive Search (Microsoft.Search/searchServices).

Metric ID Unit Description
DocumentsProcessedCount Count Total of the number of documents successfully processed in an indexing operation (either by an indexer or by pushing documents directly).
SearchLatency Seconds Average search latency for queries that execute on the search service.
SearchQueriesPerSecond CountPerSecond Average of the search queries per second (QPS) for the search service. It's common for queries to execute in milliseconds, so only queries that measure as seconds will appear in a metric like QPS.
The minimum is the lowest value for search queries per second that was registered during that minute. The same applies to the maximum value. Average is the aggregate across the entire minute. For example, within one minute, you might have a pattern like this: one second of high load that is the maximum for SearchQueriesPerSecond, followed by 58 seconds of average load, and finally one second with only one query, which is the minimum.
SkillExecutionCount Count Total number of skill executions processed during an indexer operation.
ThrottledSearchQueriesPercentage Percent Average percentage of the search queries that were throttled from the total number of search queries that executed during a one-minute interval.

For reference, see a list of all platform metrics supported in Azure Monitor.

Metric dimensions

Dimensions of a metric are name/value pairs that carry additional data to describe the metric value. Azure Cognitive Search has the following dimensions associated with its metrics that capture a count of documents or skills that were executed ("Document processed count" and "Skill execution invocation count").

Dimension Name Description
DataSourceName A named data source connection used during indexer execution. Valid values are one of the supported data source types.
Failed Indicates whether the instance failed.
IndexerName Name of an indexer.
IndexName Name of an index.
SkillsetName Name of a skillset used during indexer execution.
SkillName Name of a skill within a skillset.
SkillType The @odata.type of the skill.

For more information on what metric dimensions are, see Multi-dimensional metrics.

Resource logs

Resource logs are platform logs that provide insight into operations that were performed within an Azure resource. Resource logs are generated by the search service automatically, but are not collected by default. You must create a diagnostic setting to send resource logs to a Log Analytics workspace to use with Azure Monitor Logs, Azure Event Hubs to forward outside of Azure, or to Azure Storage for archiving.

This section identifies the type (or category) of resource logs you can collect for Azure Cognitive Search:

  • Resource logs are grouped by type (or category). Azure Cognitive Search generates resource logs under the Operations category.

For reference, see a list of all resource logs category types supported in Azure Monitor.

Azure Monitor Logs tables

Azure Monitor Logs is a feature of Azure Monitor that collects and organizes log and performance data from monitored resources. If you configured a diagnostic setting for Log Analytics, you can query the Azure Monitor Logs tables for the resource logs generated by Azure Cognitive Search.

This section refers to all of the Azure Monitor Logs Kusto tables relevant to Azure Cognitive Search and available for query by Log Analytics and Metrics Explorer in the Azure portal.

Table Description
AzureActivity Entries from the Azure Activity log that provide insight into control plane operations. Tasks invoked on the control plane, such as adding or removing replicas and partitions, will be represented through a "Get Admin Key" activity.
AzureDiagnostics Logged query and indexing operations.
AzureMetrics Metric data emitted by Azure Cognitive Search that measures health and performance.

For a reference of all Azure Monitor Logs / Log Analytics tables, see the Azure Monitor Log Table Reference.

Diagnostics tables

Azure Cognitive Search uses the Azure Diagnostics table to collect resource logs related to queries and indexing on your search service.

Queries against this table in Log Analytics can include the common properties, the search-specific properties, and the search-specific operations listed in the schema reference section.

For examples of Kusto queries useful for Azure Cognitive Search, see Monitoring Azure Cognitive Search and Analyze performance in Azure Cognitive Search.

Activity logs

The following table lists common operations related to Azure Cognitive Search that may be created in the Azure Activity log.

Operation Description
Get Admin Key Any operation that requires administrative rights will be logged as a "Get Admin Key" operation.
Get Query Key Any read-only operation against the documents collection of an index.
Regenerate Admin Key A request to regenerate either the primary or secondary admin API key.

Common entries include references to API keys - generic informational notifications like Get Admin Key and Get Query keys. These activities indicate requests that were made using the admin key (create or delete objects) or query key, but do not show the request itself. For information of this grain, you must configure diagnostic logging.

Alternatively, you might gain some insight through change history. In Azure portal, select the activity to open the detail page and then select "Change history" for information about the underlying operation.

For more information on the schema of Activity Log entries, see Activity Log schema.

Schemas

The following schemas are in use by Azure Cognitive Search. If you are building queries or custom reports, the data structures that contain Azure Cognitive Search resource logs conform to the schema below.

For resource logs sent to blob storage, each blob has one root object called records containing an array of log objects. Each blob contains records for all the operations that took place during the same hour.

Resource log schema

All resource logs available through Azure Monitor share a common top-level schema. Azure Cognitive Search supplements with additional properties and operations that are unique to a search service.

The following example illustrates a resource log that includes common properties (TimeGenerated, Resource, Category, and so forth) and search-specific properties (OperationName and OperationVersion).

Name Type Description and example
TimeGenerated Datetime Timestamp of the operation. For example: 2021-12-07T00:00:43.6872559Z
Resource String Resource ID. For example: /subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group-name>/providers/Microsoft.Search/searchServices/<your-search-service-name>
Category String "OperationLogs". This value is a constant. OperationLogs is the only category used for resource logs.
OperationName String The name of the operation (see the full list of operations below). An example is Query.Search
OperationVersion String The api-version used on the request. For example: 2020-06-30
ResultType String "Success". Other possible values: Success or Failure
ResultSignature Int An HTTP result code. For example: 200
DurationMS Int Duration of the operation in milliseconds.
Properties Object Object containing operation-specific data. See the following properties schema table.

Properties schema

The properties below are specific to Azure Cognitive Search.

Name Type Description and example
Description_s String The operation's endpoint. For example: GET /indexes('content')/docs
Documents_d Int Number of documents processed.
IndexName_s String Name of the index associated with the operation.
Query_s String The query parameters used in the request. For example: ?search=beach access&$count=true&api-version=2020-06-30

OperationName values (logged operations)

The operations below can appear in a resource log.

OperationName Description
DataSources.* Applies to indexer data sources. Can be Create, Delete, Get, List.
DebugSessions.* Applies to a debug session. Can be Create, Delete, Get, List, Start, and Status.
DebugSessions.DocumentStructure An enriched document is loaded into a debug session.
DebugSessions.RetrieveIndexerExecutionHistoricalData A request for indexer execution details.
DebugSessions.RetrieveProjectedIndexerExecutionHistoricalData Execution history for enrichments projected to a knowledge store.
Indexers.* Applies to an indexer. Can be Create, Delete, Get, List, and Status.
Indexes.* Applies to a search index. Can be Create, Delete, Get, List.
indexes.Prototype This is an index created by the Import Data wizard.
Indexing.Index This operation is a call to Add, Update or Delete Documents.
Metadata.GetMetadata A request for search service system data.
Query.Autocomplete An autocomplete query against an index. See Query types and composition.
Query.Lookup A lookup query against an index. See Query types and composition.
Query.Search A full text search request against an index. See Query types and composition.
Query.Suggest Type ahead query against an index. See Query types and composition.
ServiceStats This operation is a routine call to Get Service Statistics, either called directly or implicitly to populate a portal overview page when it is loaded or refreshed.
Skillsets.* Applies to a skillset. Can be Create, Delete, Get, List.

See also