Configure Relevance Search to improve search results and performance
Applies to Dynamics 365 for Customer Engagement apps version 9.x
Applies to Dynamics 365 for Customer Engagement apps version 9.x (on-premises)
Relevance Search delivers fast and comprehensive search results in a single list, sorted by relevance. It uses a dedicated search service external to Dynamics 365 for Customer Engagement apps powered by Azure Search to improve your Dynamics 365 for Customer Engagement apps search experience. As an administrator or customizer, you'll be able to enable and configure Relevance Search in the Customer Engagement apps user interface without writing code. Many of the configuration steps will look familiar to you, as they use the same user interface as the Quick Find configuration.
Relevance Search is available in addition to other Customer Engagement apps search experience you’re already familiar with. You can still use single-entity Quick Find on the entity grid. You can also use multi-entity Quick Find (now called Categorized Search) from the Search Dynamics 365 for Customer Engagement apps data search box on the navigation bar.
Relevance Search brings the following enhancements and benefits:
Improves performance with external indexing and Azure Search technology.
Finds matches to any word in the search term in any field in the entity. Matches may include inflectional words, like “stream,” “streaming,” or “streamed.”
Returns results from all searchable entities in a single list sorted by relevance, based on factors, such as number of words matched or their proximity to each other in the text.
Matches in the result list are highlighted.
Includes the ability to search documents found in Notes and Attachments on Emails and Appointments tracked in Customer Engagement apps.
Compare Dynamics 365 for Customer Engagement apps searches
There are three kinds of search in Dynamics 365 for Customer Engagement:
Full-text Quick Find (single-entity or multi-entity)
Quick Find (single-entity or multi-entity)
The following table provides a brief comparison of the three available searches.
|Functionality||Relevance Search||Full-Text Quick Find||Quick Find|
|Availability||Available for Dynamics 365 for Customer Engagement apps organizations that have installed Dynamics 365 for Customer Engagement apps version 9.0. Not available for Dynamics 365 for Customer Engagement apps (on-premises) organizations.||Available for Dynamics 365 for Customer Engagement apps (on-premises) organizations, starting with Dynamics CRM 2015 Update Rollup 1.||Available for Dynamics 365 for Customer Engagement apps organizations and Dynamics 365 for Customer Engagement apps (on-premises) organizations.|
|Enabled by default?||No. An administrator must manually enable it.||No. An administrator must manually enable it.||Yes|
|Single-entity search scope||Not available in an entity grid. You can filter the search results by an entity on the results page.||Available in an entity grid.||Available in an entity grid.|
|Multi-entity search scope||There is no maximum limit on the number of entities you can search. Note: While there is no maximum limit on the number of entities you can search, the Record Type filter shows data for only 10 entities.||Searches up to 10 entities, grouped by an entity.||Searches up to 10 entities, grouped by an entity.|
|Search behavior||Finds matches to any word in the search term in any field in the entity.||Finds matches to all words in the search term in one field in an entity; however, the words can be matched in any order in the field.||Finds matches as in a SQL query with “Like” clauses. You have to use the wildcard characters in the search term to search within a string. All matches must be an exact match to the search term.|
|Search results||Returns the search results in order of their relevance, in a single list.||For single-entity, returns the search results in an entity grid. For multi-entity, returns the search results grouped by categories, such as accounts, contacts, or leads.||For single-entity, returns the search results in an entity grid. For multi-entity, returns the search results grouped by categories, such as accounts, contacts, or leads.|
How Relevance Search works
Relevance Search uses the same default scoring concepts as Azure Search. Scoring refers to the computation of a search score for every item returned in search results. The score is an indicator of an item's relevance in the context of the current search operation. The higher the score, the more relevant the item. In search results, items are ranked in order from high to low, based on the search scores calculated for each item. By default, a search score is computed based on statistical properties of the data and the query. Relevance Search finds documents that include the search terms in the query string, favoring the documents that contain many instances of the words in the search term and their close proximity to each other in the document. The search score goes up even higher if the term is rare across the index, but common within the document. The results are then ranked by search score before they’re returned. Search score values can be repeated throughout a result set. For example, you might have 10 items with a score of 1.2, 20 items with a score of 1.0, and 20 items with a score of 0.5. When multiple hits have the same search score, the ordering of same-score items isn’t defined, and isn’t stable. Run the query again and you might see items shift position. Given two items with an identical score, there is no guarantee which one appears first. More information: Add scoring profiles to a search index (Azure Search Service REST API)
Searchable fields are analyzed in the Azure Search index to provide a more natural, end-user friendly search experience by breaking words into their root forms, text normalization, and filtering out noise words. All searchable fields in Relevance Search are analyzed with the Microsoft Natural language analyzer, which uses Lemmetization to break words down into their root linguistic forms. For example, “ran” will match to “run” and “running” since “run” is considered the base form of the word. Word stemmers, such as SQL full-text indexes, don’t have any linguistic context and only consider matches where the root is the same as the inflectional form. With stemming, “run” would match to “running” and “runner”, but not “ran” since it doesn’t consider “ran” to be a word linguistically related to “run”. All searchable fields in Relevance Search use an analyzer that most closely matches the organization’s base language. For Kazakh, which is the only language supported by Dynamics 365 for Customer Engagement but not by Azure Search, all fields are analyzed using the default analyzer. For more information about language analysis and a list of the supported languages, see: Language support (Azure Search Service REST API).
You'll see hit highlights when your search term matches a term in your application. These appear as bolded and italicized text in the search results. Notice these search terms are often returned as only a portion of the full value in a field since only the matched terms are highlighted. For example, using the search term L. Wendell returns the record for the contact with first name L. Wendell and last name Overby, as Wendell Overby in the search results.
Relevance Search architecture
Relevance Search is hosted on the Azure cloud computing platform and infrastructure that uses Azure Search, which provides the search results. Changes made in Customer Engagement apps may take up to 15 minutes to appear in the search service. It may take up to up to an hour or more to complete a full sync for average to large size organizations.
The following diagram shows the high level Relevance Search architecture.
Enable Relevance Search
Data in your application begins syncing to the external search index immediately after you enable Relevance Search. We strongly recommend that you configure the entities and entity fields participating in Relevance Search before you enable the search, to prevent sensitive data from being indexed in a service external to Dynamics 365 for Customer Engagement apps. For more information about configuring Relevance Search, see Select entities for Relevance Search, Configure searchable fields for Relevance Search, and Set managed property for Relevance Search.
Because you’ll be sharing your Customer Engagement apps data with the external system, Relevance Search is disabled by default. To enable it, you must accept the consent terms. Depending on the size of your organization, it may take up to an hour or more for the data to become available in the external search index after you enable the search. Enabling Relevance Search makes this search option available to all members of your organization.
By default, Relevance Search is disabled. To enable Relevance Search, do the following:
Go to Settings > Administration.
Click System Settings > General tab.
In the Set up Search sub-area, select the Enable Relevance Search check box, as shown here.
After you enable Relevance Search, the Enable Search consent dialog box opens. Click OK to give your consent.
Click OK to close the System Settings dialog.
Select entities for Relevance Search
To configure Relevance Search, use the Configure Relevance Search selection on the task bar, as shown here.
There is no limit on how many entities you can include in the Relevance Search results. However, there is a limit on the total number of fields in the external search index. Currently, the maximum is 1000 searchable fields for an organization. When you select an entity to include in the search results, you’ll notice a number in parentheses next to the entity name. The number indicates how many fields each entity uses in the external search index. Some fields, such as Primary Name and ID, are shared by multiple entities and don't count toward the total. Additionally, some field types use more than one field in the external search index as indicated in this table.
|Field type||Number of fields used in the external search index|
|Lookup (customer, owner, or Lookup type attribute)||3|
|Option Set (state, or status type attribute)||2|
|All other types of fields||1|
The progress bar Total fields indexed shows the percentage of indexed fields to the maximum allowed number of searchable fields.
When you have reached the indexed field limit, you’ll see a warning message. If you want to add more fields to the index, you’ll have to free up space, either by removing some of the fields that are already in the index or removing entire entities from Relevance Search.
To select entities for the Relevance Search results, do the following:
Go to Settings > Customizations.
Click Customize the System.
Under Components, expand Entities, and then click Configure Relevance Search.
The Select Entities dialog box opens. Click Add to select the entities for the search results. When you’re done, click OK.
Click Publish All Customizations for your changes to take effect.
By default, some out-of-the-box system entities are included in Relevance Search. However, custom entities aren’t included. You have to add them to Relevance Search.
Configure searchable fields for Relevance Search
The fields you add in the Quick Find view become part of the external search index. There is no limit on how many searchable fields you can add for each entity. However, there is a limit on the total number of indexed fields, as was explained in the previous section. Find Columns on a Quick Find View define the searchable fields in the external search index. Text fields such as Single Line of Text and Multiple Lines of Text, Lookups, and Option Sets are searchable. Find Columns with other data types are ignored. The View Columns on a Quick Find View define the fields that are displayed in the user interface by default, when the matched results are returned. The fields that are highlighted replace the fields that don’t have the highlighting. The first four matched fields are displayed in the results. The filter on a Quick Find view is also applied to the Relevance Search results. See the table below for the list of filter clauses not supported by Relevance Search.
There are some fields, called common fields, common to every CRM entity that are defined on the index by default. They are:
- ownerid (Name of lookup)
- owningbusinessunit (Name of lookup)
- statecode (Label of optionset)
- statuscode (Label of optionset)
- name (Primary name field of any entity. This may or may not be the same as the logical name (fullname, subject etc.) of the entity) If a common field is added to any entity for Relevance Search, search will be performed for that common field across all entities. However, once you choose a specific entity through the Record Type facet, Relevance Search will follow the settings you have defined for that specific entity through Quick Find View.
You can use the Quick Find view to define which fields appear as facets when users search by using Relevance Search. All View Columns with data types other than Single Line of Text and Multiple Lines of Text are marked as facetable and filterable in the index. By default, the first four facetable fields in the Quick Find view for the selected entity are displayed as facets when users search by using Relevance Search. At any time, you can only have four fields selected as facets.
Go to Settings > Customizations.
Click Customize the System.
Under Components, expand Entities, and then expand the entity you want.
In the navigation tree, click View. Double-click Quick Find View. The following illustration shows the Quick Find view for the
Click Add Find Columns. In the dialog box, select the fields you want to add to the search index. When done, click OK. In the following illustration, you see the
Accountentity fields added to the external search index.
Repeat the steps for the View Columns.
Click Publish All Customizations for your changes to take effect.
The changes you make in Quick Find view also apply to single-entity and multi-entity (Categorized Search) Quick Find configurations. This is why we don't prevent you from including the fields that aren't supported for Relevance Search when you configure Quick Find view. However, unsupported fields aren't synced to the external index and don’t appear in the Relevance Search results.
For Relevance Search, fields on a related entity are not supported as Find, View, or Filter fields.
The following table contains the Quick Find Filter operators that aren’t supported for Relevance Search:
|Dynamics 365 for Customer Engagement apps Operator|
Set managed property for Relevance Search
If you want to include an entity in Relevance Search, the Can enable sync to external search index managed property for this entity must be set to True. By default, the property is set to True for some of the out-of-the-box system entities and all custom entities. Some of the system entities can’t be enabled for Relevance Search.
To set the managed property, do the following:
Go to Settings > Customizations.
Click Customize the System.
Under Components, expand Entities, and then click the entity you want.
On the menu bar, click Managed Properties. For Can enable sync to external search index, click True or False to set the property to the desired state. Click Set to exit, as shown here.
Click Publish for your changes to take effect.
If you want to change the Can enable sync to external search index property to False, you must first deselect the entity from Relevance search. If the entity is included in Relevance Search, you’ll see the following message: “This entity is currently syncing to an external search index. You must remove the entity from the external search index before you can set the Can Enable Sync to External Search Index property to False.” If Can Enable Sync to External Search Index is set to False, you’ll see the following message when you try to include an entity in Relevance Search: “Entity can’t be enabled for Relevance Search because of the configuration of its managed properties.” For custom entities with particularly sensitive data, you may consider setting the Can enable sync to external search index property to False. Keep in mind, after you install the managed solution on the target system, you won’t be able to change the value of the property because it’s a managed property.
By enabling Relevance Search, data in participating entities and attributes in your Dynamics 365 for Customer Engagement instance will begin syncing to and be stored in an Azure Search index.
Relevance Search is not enabled by default. The system administrator must enable the functionality within a Dynamics 365 for Customer Engagement instance. After Relevance Search is enabled, system administrators and customizers have full control over the data that will be synchronized to the Azure Search index.
System customizers can use the Configure Relevance Search dialog box in Customization Tools to enable specific entities for search and then configure Quick Find views on enabled entities to select the searchable attributes. Data changes are synchronized continuously between Dynamics 365 for Customer Engagement and Azure Search through a secure connection. Configuration data is encrypted and the required secrets are stored in Azure Key Vault.
Azure components and services that are involved with Relevance Search functionality are detailed in the following sections.
Microsoft Azure Trust Center
An Azure Search index is used to provide high-quality search results with quick response times. Azure Search adds powerful and sophisticated next-generation search capabilities to Dynamics 365 for Customer Engagement. This is a dedicated search service external to Dynamics 365 for Customer Engagement provided by Azure. All new Azure Search indexes are encrypted at rest. If you opted in before January 24, 2018, you'll need to reindex your data by opting out of Relevance Search, waiting an hour, and opting back in.
Relevance Search uses the Azure SQL Database to store:
Configuration data related to the organization and the corresponding index
Metadata relating to the search service and indexes
Pointers to system metadata and data when synchronizing changes
Authorization data to enable enhanced row- level security
The Azure Event Hubs component is used for message exchange between Dynamics 365 for Customer Engagement and Azure and to maintain work items that are managed by the synchronization process. Each message stores information, such as the organization ID and entity name, used to sync the data.
The processing and indexing of data is handled in micro-services deployed on virtual machines managed through the Service Fabric runtime. The search APIs and the data synchronization process are also hosted on the Service Fabric cluster.
Service Fabric was born from years of experience at Microsoft delivering mission-critical cloud services and is now production-proven for over five years. It’s the foundational technology on which we run our Azure core infrastructure, powering services including Skype for Business, Intune, Azure Event Hubs, Azure Data Factory, Azure DocumentDB, Azure SQL Database, and Cortana—which can scale to process more than 500 million evaluations per second.
Azure Virtual Machine Scale Sets are elastic and designed to support hyper scale-out workloads. The Azure Service Fabric cluster runs on virtual machine scale sets. The micro-services for processing and indexing data are hosted on the scale sets and managed by the Service Fabric runtime.
Azure Key Vault is used for secure management of certificates, keys, and other secrets used in the search process.
Changes to customer data are stored for up to 2 days in Azure Blob Storage. These blobs are encrypted by leveraging the latest feature in the Azure Storage SDK, which provides symmetric and asymmetric encryption support and integration with Azure Key Vault. With the December 2016 update for Dynamics 365 (online), the documents found in Notes and Attachments on email messages and appointments are also synced to the blob storage.
Azure Active Directory is used to authenticate between the Dynamics 365 for Customer Engagement and Azure services.
The Azure Load Balancer is used to distribute incoming traffic among healthy service instances in cloud services or virtual machines defined in a load balancer set. Relevance Search uses it to load balance the end points in a deployment.
The Virtual Machines on the Service Fabric cluster running in one or more subnets are connected by Azure Virtual Network. The security policies, DNS settings, route tables, and IP addresses are fully controlled within this virtual network. Network Security Groups are leveraged to apply security rules on this virtual network. These rules allow or deny network traffic to the VMs in the virtual network.