What's new in Azure AI Search
Azure Cognitive Search is now Azure AI Search. Learn about the latest updates to Azure AI Search functionality, docs, and samples.
Note
Preview features are announced here, but we also maintain a preview features list so you can find them in one place.
May 2024
| Item | Type | Description |
|---|---|---|
| Higher capacity and more vector quota at every tier (same billing rate) | Infrastructure | Partition sizes are now even larger for Standard 2 (S2), Standard 3 (S3), and Standard 3 High Density (S3 HD) for all services created after April 3, 2024. If you create a new service now, you get the larger partitions. If you created a new service between April 3 and May 17, you get the larger partitions automatically. Storage Optimized tiers (L1 and L2) also have more capacity. L1 and L2 customers must create a new service to benefit from the higher capacity. There's no in-place upgrade at this time. Extra capacity is now available in more regions: Germany North, Germany West Central, South Africa North, Switzerland West, and Azure Government (Texas, Arizona, and Virginia). |
| OneLake files and shortcuts integration (preview) | Feature | New indexer for OneLake files and OneLake shortcuts. If you use Microsoft Fabric and OneLake for data access to Amazon Web Services (AWS) and Google data sources, use this indexer to import external data into a search index. This indexer is available through the Azure portal, the 2024-05-01-preview REST API, and Azure SDK beta packages. |
| Relevance tuning and search results customization | Feature | Three enhancements improve vector search relevance. First, you can now set thresholds on vector search results to exclude low-scoring results. Second, you can set MaxSizeTextRecall and countAndFacetMode in hybrid queries to specify the maximum number of documents that can be recalled using text query in hybrid (text and vector) search. Previously, the maximum was fixed at 1,000. If you have more matches, you can now specify a higher limit to get more results back. Third, for hybrid queries, you can set a weight on vector queries to have more or less importance than the nonvector query. |
| Binary vectors support | Feature | Collection(Edm.Byte) is a new supported data type. This data type opens up integration with the Cohere v3 binary embedding models and custom binary quantization. Narrow data types lower the cost of large vector datasets. See Index binary data for vector search for more information. |
| Azure AI Vision multimodal embeddings skill (preview) | Skill | New skill that's bound to the multimodal embeddings API of Azure AI Vision. You can generate embeddings for text or images during indexing. This skill is available through the Azure portal and the 2024-05-01-preview REST API. |
| Azure AI Vision vectorizer (preview) | Vectorizer | New vectorizer connects to an Azure AI Vision resource using the multimodal embeddings API to generate embeddings at query time. This vectorizer is available through the Azure portal and the 2024-05-01-preview REST API. |
| Azure AI Studio model catalog vectorizer (preview) | Vectorizer | New vectorizer connects to an embedding model deployed from the Azure AI Studio model catalog. This vectorizer is available through the Azure portal and the 2024-05-01-preview REST API. How to implement integrated vectorization using models from Azure AI Studio. |
| AzureOpenAIEmbedding skill (preview) supports more models on Azure OpenAI | Skill | Updates to this skill add support for more embedding models on Azure OpenAI. New dimensions and modelName properties are used for specifying models. Previously, the dimensions limits were fixed at 1,536 dimensions. It's now configurable. This update is available through the Azure portal and the 2024-05-01-preview REST API. |
| 2024-05-01-preview Search REST API | API | New preview version of the Search REST APIs provides new skills and vectorizers, new binary data type, OneLake files indexer, and new query parameters for more relevant results. See Upgrade REST APIs if you have existing code written against the 2023-07-01-preview and need to migrate to this version. |
| Azure SDK beta packages for new features | API | Review the changelogs of the following Azure SDK beta packages for new feature support: Azure SDK for Python, Azure SDK for .NET, Azure SDK for Java |
| Python code samples | Samples | New end-to-end samples demonstrate integration with Cohere Embed v3, integration with OneLake and cloud data platforms on Google and AWS, and integration with Azure AI Vision multimodal APIs. |
April 2024
| Item | Type | Description |
|---|---|---|
| Security update addressing information disclosure | API | GET responses no longer return connection strings or keys. Applies to GET Skillset, GET Index, and GET Indexer. This change helps protect your Azure assets integrated with AI Search from unauthorized access. |
| More storage on Basic and Standard tiers | Infrastructure | Basic now supports up to three partitions and three replicas. Basic and Standard (S1, S2, S3) tiers have significantly more storage per partition, at the same per-partition billing rate. Extra capacity is subject to regional availability and applies to new search services created after April 3, 2024. Currently, there's no in-place upgrade, so please create a new search service to get the extra storage. |
| More quota for vectors | Infrastructure | Vector quotas are also higher on new services created after April 3, 2024 in selected regions. |
Vector quantization, narrow vector data types, and a new stored property (preview) |
Feature | Collectively, these three features add vector compression and smarter storage options. First, scalar quantization reduces vector index size in memory and on disk. Second, narrow data types reduce per-field storage by storing smaller values. Third, you can use stored to opt-out of storing the extra copy of a vector that's used only for search results. If you don't need vectors in a query response, you can set stored to false to save on space. |
| 2024-03-01-preview Search REST API | API | New preview version of the Search REST APIs for the new data types, vector compression properties, and vector storage options. |
| 2024-03-01-preview Management REST API | API | New preview version of the Management REST APIs for control plane operations. |
| 2023-07-01-preview deprecation announcement | API | Deprecation announced on April 8, 2024. Retirement on July 8, 2024. This was the first REST API that offered vector search support. Newer API versions have a different vector configuration. We recommend migrating to a newer version as soon as possible. |
February 2024
| Item | Type | Description |
|---|---|---|
| New dimension limits | Feature | For vector fields, maximum dimension limits are now 3072, up from 2048. |
2023 announcements
| Month | Type | Announcement |
|---|---|---|
| November | Feature | Vector search, generally available. The previous restriction on customer-managed keys (CMK) is now lifted. Prefiltering and exhaustive K-nearest neighbor algorithm are also now generally available. |
| November | Feature | Semantic ranking, generally available |
| November | Feature | Integrated vectorization (preview) adds data chunking and text-to-vector conversions during indexing, and also adds text-to-vector conversions at query time. |
| November | Feature | Import and vectorize data wizard (preview) automates data chunking and vectorization. It targets the 2023-10-01-Preview REST API. |
| November | Feature | Index projections (preview) defines the shape of a secondary index, used for a one-to-many index pattern, where content from an enrichment pipeline can target multiple indexes. |
| November | API | 2023-11-01 Search REST API is stable version of the Search REST APIs for vector search and semantic ranking. See Upgrade REST APIs for migration steps to generally available features. |
| November | API | 2023-11-01 Management REST API adds APIs that enable or disable semantic ranking. |
| November | Skill | Azure OpenAI Embedding skill (preview) connects to a deployed embedding model on your Azure OpenAI resource to generate embeddings during skillset execution. |
| November | Skill | Text Split skill (preview) updated in 2023-10-01-Preview to support native data chunking. |
| November | Video | How vector search and semantic ranking improve your GPT prompts explains how hybrid retrieval gives you optimal grounding data for generating useful AI responses and enables search over both concepts and keywords. |
| November | Sample | Role-based access control in Generative AI applications explains how to use Microsoft Entra ID and Microsoft Graph API to roll out granular user permissions on chunked content in your index. |
| October | Sample | "Chat with your data" solution accelerator. End-to-end RAG pattern that uses Azure AI Search as a retriever. It provides indexing, data chunking, and orchestration. |
| October | Feature | Exhaustive K-Nearest Neighbors (KNN) scoring algorithm for similarity search in vector space. Available in the 2023-10-01-Preview REST API only. |
| October | Feature | Prefilters in vector search evaluate filter criteria before query execution, reducing the amount of content that needs to be searched. Available in the 2023-10-01-Preview REST API only, through a new vectorFilterMode property on the query that can be set to preFilter (default) or postFilter, depending on your requirements. |
| October | API | 2023-10-01-Preview Search REST API, breaking changes the definition for vector fields and vector queries. |
| August | Feature | Enhanced semantic ranking. Upgraded models are rolling out for semantic reranking, and availability is extended to more regions. Maximum unique token counts doubled from 128 to 256. |
| July | Sample | Vector demo (Azure SDK for JavaScript). Uses Node.js and the @azure/search-documents 12.0.0-beta.2 library to generate embeddings, create and load an index, and run several vector queries. |
| July | Sample | Vector demo (Azure SDK for .NET). Uses the Azure.Search.Documents 11.5.0-beta.3 library to generate embeddings, create and load an index, and run several vector queries. You can also try this sample from the Azure SDK team. |
| July | Sample | Vector demo (Azure SDK for Python) Uses the latest beta release of the azure.search.documents to generate embeddings, create and load an index, and run several vector queries. Visit the azure-search-vector-samples/demo-python repo for more vector search demos. |
| June | Feature | Vector search public preview. |
| June | Feature | Semantic search availability, available on the Basic tier. |
| June | API | 2023-07-01-Preview Search REST API. Support for vector search. |
| May | Feature | Azure RBAC (role-based access control, generally available). |
| May | API | 2022-09-01 Management REST API, with support for configuring search to use Azure roles. The Az.Search module of Azure PowerShell and Az search module of the Azure CLI are updated to support search service authentication options. You can also use the Terraform provider to configure authentication options (see this Terraform quickstart for details). |
| April | Sample | Multi-region deployment of Azure AI Search for business continuity and disaster recovery. Deployment scripts that fully configure a multi-regional solution for Azure AI Search, with options for synchronizing content and request redirection if an endpoint fails. |
| March | Sample | ChatGPT + Enterprise data with Azure OpenAI and Azure AI Search (GitHub). Python code and a template for combining Azure AI Search with the large language models in OpenAI. For background, see this Tech Community blog post: Revolutionize your Enterprise Data with ChatGPT. Key points: Use Azure AI Search to consolidate and index searchable content. Query the index for initial search results. Assemble prompts from those results and send to the gpt-35-turbo (preview) model in Azure OpenAI. Return a cross-document answer and provide citations and transparency in your customer-facing app so that users can assess the response. |
Previous year's announcements
Service rebrand
This service has had multiple names over the years. Here they are in reverse chronological order:
- Azure AI Search (November 2023) Renamed to align with Azure AI services and customer expectations.
- Azure Cognitive Search (October 2019) Renamed to reflect the expanded (yet optional) use of cognitive skills and AI processing in service operations.
- Azure Search (March 2015) The original name.
Service updates
Service update announcements for Azure AI Search can be found on the Azure web site.
Feature rename
Semantic search was renamed to semantic ranking in November 2023 to better describe the feature, which provides L2 ranking of an existing result set.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for