Skip to content

Duplication Check Configuration

Administrators can define rules that automatically flag potential duplicate entities within your datasets. By specifying which properties to compare and how their values should match, the system identifies potential duplicates during entity editing and validation.

Accessing the duplication check configuration

You can reach the settings by clicking the gear icon in the sidebar to open the Admin view, then selecting the tab * DUPLICATION CHECK*.

Note: This section is accessible only to users with ADMIN role.

AI Matcher

At the top of the page, you can enable the toggle AI Matcher for duplication check. The AI Matcher uses AI-based similarity to automatically detect potential duplicates and can be used on its own. If you need stricter control or domain-specific criteria, define your own rules to complement the AI Matcher.

Create a Rule

  1. Click the plus icon in the table toolbar (or Add rule).
  2. Enter a descriptive name (e.g., "Product name duplicates").
  3. Select the dataset(s) where the rule applies (at least one is required).
  4. Optionally select specific object types to limit the scope. If left empty, all entity types are checked.
  5. Define matching conditions:
    • Add one or more property groups with Add property group.
    • Inside a group, select one or more properties. Within a group, properties are evaluated with OR logic (any selected property can match).
    • Choose the comparison operator for the group:
      • equal: exact match
      • similar: approximate match (fuzzy matching)
    • Across groups, conditions are evaluated with AND logic (all groups must match).
  6. Click Save.

Important: Duplicate checks are performed only within the same dataset. When multiple datasets are selected, each dataset is evaluated independently; there is no cross-dataset comparison.

After saving, the rule becomes active immediately and is used when working with entities in the selected datasets.

How matching works

Rules compare property values of the current entity against other entities in the same dataset and follow these rules:

  • Within a group (OR):
    • The group matches if any selected property on the candidate shares at least one value that is equal/similar (based on the chosen operator) to a value on the current entity.
  • Across groups (AND):
    • If you add multiple groups, every group must match the same candidate entity for it to be flagged.

Examples:

1) Single group with equal

  • Group 1: properties [Label, Name], operator equal
  • Current entity values: Label = "Aspirin", Name = "Acetylsalicylic acid"
  • A candidate matches if it has Label = ["Aspirin" OR "Acetylsalicylic acid"] OR Name = ["Acetylsalicylic acid" OR "Aspirin"].

2) Two groups (AND across groups)

  • Group 1: properties [Label, Name], operator equal
  • Group 2: properties [Code], operator similar
  • Current entity values: Label = "Aspirin", Name = "Acetylsalicylic acid", Code = "123-456"
  • A candidate is flagged only if Group 1 matches (Label = ["Aspirin" OR "Acetylsalicylic acid"] OR Name = ["Acetylsalicylic acid" OR "Aspirin"]) AND Group 2 matches (Code similar to "123-456", e.g., "123456").

Where duplicate warnings appear

  • While creating or editing an entity in the Global Entity View: a banner at the top of the page shows potential duplicates.
  • In validation: the Pending Changes Validation dialog includes a section Possible Duplicates.

For each potential duplicate, Accurids shows:

  • A similarity score (0% – 100%; higher means more similar)
  • The detection source (a specific rule or the AI Matcher)
  • A compare and merge button (when merge is available) to merge the duplicate. Read more

Rule-based matches appear first in the list, followed by AI Matcher suggestions. This prioritization helps you review explicit, administrator-defined criteria before AI-suggested candidates.