Detect duplicate data


Applies To: Dynamics CRM 2013

Duplicate detection lets organizations set duplicate detection policies and create duplicate detection rules for business and custom entities. These rules can be applied across different record types in Microsoft Dynamics CRM. For example, an organization may define that a lead is a duplicate of a contact, if they have the same name and phone number. Based on the duplicate detection rules set by the administrator, the system alerts the user about potential duplicates when the user tries to create new records or update existing records. To maintain data quality, you can schedule a duplicate detection job to check for duplicates for all records that match a certain criteria. You can clean the data by deleting, deactivating, or merging the duplicates reported by a duplicate detection job.


The Microsoft Dynamics CRM Online Spring ‘14 and Microsoft Dynamics CRM 2013 Service Pack 1 (SP1) bring back the capability to detect duplicates for the updated user interface (UI) entities, when you create or update a record using entity forms or grid views in the CRM web application. The entities must be enabled for duplicate detection.

Also, new in Microsoft Dynamics CRM Online Spring ‘14 and Microsoft Dynamics CRM 2013 Service Pack 1 (SP1) is the capability to detect duplicates in CRM for tablets.

To detect duplicates in the system, create a duplicate detection rule for a specific entity type. A duplicate detection rule is represented by the duplicate rule (DuplicateRule) entity. You can create multiple detection rules for the same entity type. However, you can publish a maximum of five duplicate detection rules per entity type at one time.

A rule can have one or more duplicate detection rule conditions that are represented by the duplicate rule condition (DuplicateRuleCondition) entity. The conditions are combined by the system as in logical AND operation. A duplicate detection rule specifies a base entity type and a matching entity type. A duplicate rule condition specifies the name of a base attribute and the name of a matching attribute. For example, specify an account as a base entity and a contact as a matching entity to compare last names and addresses. The matching criteria consist of operators such as exactly match, first n-number of characters, or last n-number of characters.

To create duplicate detection rules and duplicate detection rule conditions in the Microsoft Dynamics CRM database, use the IOrganizationService.Create method or the CreateRequest message.


Take special precautions when you create duplicate detection rules for appointments. The recurring appointment master and the recurring appointment records share some of the same attribute values, such as subject, location, and regarding object. Using any of these attributes as a duplicate detection rule condition (DuplicateRuleCondition), may result in incorrectly marking recurring appointment records as duplicates. To avoid this, also include a start time attribute as a duplicate detection rule condition. For more information about working with recurring appointments, see Schedule and appointment entities.

The duplicate detection rules are system-wide. You must publish them before running a duplicate detection job to detect duplicates for bulk data or retrieve duplicates for a particular entity record. To publish a duplicate detection rule, use the PublishDuplicateRuleRequest message. Duplicate rule publishing is an asynchronous operation that runs in the background.

In This Section

Enable duplicate detection

Run duplicate detection

Duplicate detection messages

Ignore blank values and inactive records in duplicate detection

Disable duplicate detection

DuplicateRule entity messages and methods

DuplicateRuleCondition entity messages and methods

DuplicateRecord entity messages and methods

Sample: Enable duplicate detection and retrieve duplicates

Sample: Invoke duplicate detection for creating and updating records

Sample: Detect multiple duplicate records

Manage your data

Delete data in bulk