Azure Cognitive Search monitoring data reference

Članak
07/11/2022
6 min. za čitanje

This article provides a reference of log and metric data collected to analyze the performance and availability of Azure Cognitive Search. See Monitoring Azure Cognitive Search for an overview.

Metrics

This section lists the platform metrics collected for Azure Cognitive Search (Microsoft.Search/searchServices).

Metric ID	Unit	Description
DocumentsProcessedCount	Count	Total of the number of documents successfully processed in an indexing operation (either by an indexer or by pushing documents directly).
SearchLatency	Seconds	Average search latency for queries that execute on the search service.
SearchQueriesPerSecond	CountPerSecond	Average of the search queries per second (QPS) for the search service. It's common for queries to execute in milliseconds, so only queries that measure as seconds will appear in a metric like QPS. The minimum is the lowest value for search queries per second that was registered during that minute. The same applies to the maximum value. Average is the aggregate across the entire minute. For example, within one minute, you might have a pattern like this: one second of high load that is the maximum for SearchQueriesPerSecond, followed by 58 seconds of average load, and finally one second with only one query, which is the minimum.
SkillExecutionCount	Count	Total number of skill executions processed during an indexer operation.
ThrottledSearchQueriesPercentage	Percent	Average percentage of the search queries that were throttled from the total number of search queries that executed during a one-minute interval.

For reference, see a list of all platform metrics supported in Azure Monitor.

Metric dimensions

Dimensions of a metric are name/value pairs that carry additional data to describe the metric value. Azure Cognitive Search has the following dimensions associated with its metrics that capture a count of documents or skills that were executed ("Document processed count" and "Skill execution invocation count").

Dimension Name	Description
DataSourceName	A named data source connection used during indexer execution. Valid values are one of the supported data source types.
Failed	Indicates whether the instance failed.
IndexerName	Name of an indexer.
IndexName	Name of an index.
SkillsetName	Name of a skillset used during indexer execution.
SkillName	Name of a skill within a skillset.
SkillType	The @odata.type of the skill.

For more information on what metric dimensions are, see Multi-dimensional metrics.

Resource logs

Resource logs are platform logs that provide insight into operations that were performed within an Azure resource. Resource logs are generated by the search service automatically, but are not collected by default. You must create a diagnostic setting to send resource logs to a Log Analytics workspace to use with Azure Monitor Logs, Azure Event Hubs to forward outside of Azure, or to Azure Storage for archiving.

This section identifies the type (or category) of resource logs you can collect for Azure Cognitive Search:

Resource logs are grouped by type (or category). Azure Cognitive Search generates resource logs under the Operations category.

For reference, see a list of all resource logs category types supported in Azure Monitor.

Azure Monitor Logs tables

Azure Monitor Logs is a feature of Azure Monitor that collects and organizes log and performance data from monitored resources. If you configured a diagnostic setting for Log Analytics, you can query the Azure Monitor Logs tables for the resource logs generated by Azure Cognitive Search.

This section refers to all of the Azure Monitor Logs Kusto tables relevant to Azure Cognitive Search and available for query by Log Analytics and Metrics Explorer in the Azure portal.

Table	Description
AzureActivity	Entries from the Azure Activity log that provide insight into control plane operations. Tasks invoked on the control plane, such as adding or removing replicas and partitions, will be represented through a "Get Admin Key" activity.
AzureDiagnostics	Logged query and indexing operations.
AzureMetrics	Metric data emitted by Azure Cognitive Search that measures health and performance.

For a reference of all Azure Monitor Logs / Log Analytics tables, see the Azure Monitor Log Table Reference.

Diagnostics tables

Azure Cognitive Search uses the Azure Diagnostics table to collect resource logs related to queries and indexing on your search service.

Queries against this table in Log Analytics can include the common properties, the search-specific properties, and the search-specific operations listed in the schema reference section.

For examples of Kusto queries useful for Azure Cognitive Search, see Monitoring Azure Cognitive Search and Analyze performance in Azure Cognitive Search.

Activity logs

The following table lists common operations related to Azure Cognitive Search that may be created in the Azure Activity log.

Operation	Description
Get Admin Key	Any operation that requires administrative rights will be logged as a "Get Admin Key" operation.
Get Query Key	Any read-only operation against the documents collection of an index.
Regenerate Admin Key	A request to regenerate either the primary or secondary admin API key.

Common entries include references to API keys - generic informational notifications like Get Admin Key and Get Query keys. These activities indicate requests that were made using the admin key (create or delete objects) or query key, but do not show the request itself. For information of this grain, you must configure resource logging.

Alternatively, you might gain some insight through change history. In Azure portal, select the activity to open the detail page and then select "Change history" for information about the underlying operation.

For more information on the schema of Activity Log entries, see Activity Log schema.

Schemas

The following schemas are in use by Azure Cognitive Search. If you are building queries or custom reports, the data structures that contain Azure Cognitive Search resource logs conform to the schema below.

For resource logs sent to blob storage, each blob has one root object called records containing an array of log objects. Each blob contains records for all the operations that took place during the same hour.

Resource log schema

All resource logs available through Azure Monitor share a common top-level schema. Azure Cognitive Search supplements with additional properties and operations that are unique to a search service.

The following example illustrates a resource log that includes common properties (TimeGenerated, Resource, Category, and so forth) and search-specific properties (OperationName and OperationVersion).

Name	Type	Description and example
TimeGenerated	Datetime	Timestamp of the operation. For example: `2021-12-07T00:00:43.6872559Z`
Resource	String	Resource ID. For example: `/subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group-name>/providers/Microsoft.Search/searchServices/<your-search-service-name>`
Category	String	"OperationLogs". This value is a constant. OperationLogs is the only category used for resource logs.
OperationName	String	The name of the operation (see the full list of operations below). An example is `Query.Search`
OperationVersion	String	The api-version used on the request. For example: `2020-06-30`
ResultType	String	"Success". Other possible values: Success or Failure
ResultSignature	Int	An HTTP result code. For example: `200`
DurationMS	Int	Duration of the operation in milliseconds.
Properties	Object	Object containing operation-specific data. See the following properties schema table.

Properties schema

The properties below are specific to Azure Cognitive Search.

Name	Type	Description and example
Description_s	String	The operation's endpoint. For example: `GET /indexes('content')/docs`
Documents_d	Int	Number of documents processed.
IndexName_s	String	Name of the index associated with the operation.
Query_s	String	The query parameters used in the request. For example: `?search=beach access&$count=true&api-version=2020-06-30`

OperationName values (logged operations)

The operations below can appear in a resource log.

OperationName	Description
DataSources.*	Applies to indexer data sources. Can be Create, Delete, Get, List.
DebugSessions.*	Applies to a debug session. Can be Create, Delete, Get, List, Start, and Status.
DebugSessions.DocumentStructure	An enriched document is loaded into a debug session.
DebugSessions.RetrieveIndexerExecutionHistoricalData	A request for indexer execution details.
DebugSessions.RetrieveProjectedIndexerExecutionHistoricalData	Execution history for enrichments projected to a knowledge store.
Indexers.*	Applies to an indexer. Can be Create, Delete, Get, List, and Status.
Indexes.*	Applies to a search index. Can be Create, Delete, Get, List.
indexes.Prototype	This is an index created by the Import Data wizard.
Indexing.Index	This operation is a call to Add, Update or Delete Documents.
Metadata.GetMetadata	A request for search service system data.
Query.Autocomplete	An autocomplete query against an index. See Query types and composition.
Query.Lookup	A lookup query against an index. See Query types and composition.
Query.Search	A full text search request against an index. See Query types and composition.
Query.Suggest	Type ahead query against an index. See Query types and composition.
ServiceStats	This operation is a routine call to Get Service Statistics, either called directly or implicitly to populate a portal overview page when it is loaded or refreshed.
Skillsets.*	Applies to a skillset. Can be Create, Delete, Get, List.