Contact Deduplication Without Losing History

One person exists in your CRM seven different ways. They signed up with their work email, then their personal email, then your form captured them as a lead with a typo in the email, then your API sync brought them in again, then someone manually entered them. Seven records. One person. Your lead scoring model sees seven weaker signals instead of one clear story.

Why duplicate records wreck your data and workflows

Duplicates cause three kinds of damage. First, they break engagement tracking. One email address shows 3 opens, the second shows 2, the third shows 0. The AI model sees three people with weak engagement history instead of one person with six total opens. Second, they break suppression lists. You add john@company.com to your suppress list, but john@company.com (second record) is not on it, so you mail him twice. Third, they break workflows. A workflow that triggers on form submission might run for 10 duplicate records of the same person, sending them the same email sequence five times.

Deduplication is not data cleanup. It is restoration of ground truth.

How duplicates form in the first place

Most duplicates come from three sources. First, forms and APIs bringing in the same person under different identifiers: a contact form captures firstname + lastname, an API sync brings them as email + phone, an import CSV uses a different email format. Second, manual entry: sales reps entering leads without checking if they already exist. Third, legitimate role changes: someone changes emails or companies, and instead of merging their old and new records, both stay in the system.

Prevention is easier than cleanup, but you have to start cleanup now before the duplicates age and splinter across a dozen integrations.

Finding duplicates: the three matching strategies

Exact match is the easiest: two records with identical email addresses. Most CRMs have a built-in deduplicate function that catches these. If your CRM is a spreadsheet, sort by email and look for exact matches.

Fuzzy match is harder. john.smith@company.com and john_smith@company.com are the same person. John Smith and Jon Smith are probably the same. The matching algorithm has to tolerate small variations. Levenshtein distance (a measure of how many character edits separate two strings) works here. A distance of 1 or 2 usually means the same person.

• Exact matches: email, phone, or company ID must be identical
• Close fuzzy matches: email differs by 1 character, or first name and last name are swapped
• Weak fuzzy matches: email domain and last name match but first name is missing or abbreviated

Do not merge on weak fuzzy matches. You will destroy real people. Merge only on exact matches and very close fuzzy matches.

Finding duplicates without a dedup tool

If your CRM has a dedup function, use it. HubSpot, Pipedrive, Salesforce all have built-in tools. If you use a spreadsheet or a CSV-based system, you need a manual or semi-manual approach.

Export your contact list to Google Sheets. Use a formula to find duplicates by email: =COUNTIF($A:$A,A2)>1. This flags any email that appears more than once. Sort by this flag and review the flagged rows.

For each group of potential duplicates, check the engagement history manually. If both records show activity (email opens, form submissions, meetings) and the activities are from the same timeframe, they are probably the same person and one is a duplicate. If activities are from different years, they might be the same person at different jobs, and you should merge them with a note.

Merging without losing the story

When you merge two duplicate records, the CRM should combine their engagement histories, keep the better contact information, and flag any merged data as coming from multiple sources. This matters because you want to preserve the narrative: this person was engaged two years ago, went silent, then engaged again six months ago.

The merge process looks like this: (1) Choose the record to keep — usually the one with the most recent activity. (2) Assign all activity and relationships from the duplicate record to the keeper record. (3) Delete the duplicate record. (4) Add a note to the keeper that data was merged on [date].

HubSpot and Pipedrive handle this automatically. If you are using a spreadsheet or manual system, write a note in the keeper record identifying what was merged from where.

The merge that preserves everything

Some CRMs support soft merging: combining the records logically without deleting the source. Salesforce allows this with account hierarchies. If your tool does not support soft merge, you can approximate it by linking the old record to the new one with a relationship field saying master=keeper-record-id.

This matters if you ever need to audit the merge. You want to be able to show: we combined record A and record B into record A, and here is where all the data came from.

Preventing duplicates so you do not repeat this

After cleanup, enforce these rules: (1) Forms and APIs should match incoming contacts against existing ones before creating a new record. Check email, phone, and company+name. (2) If a high-confidence match exists, update it instead of creating a new record. (3) If a low-confidence match exists (same last name and company but no email), queue it for manual review instead of auto-creating. (4) Train your team: do not manually create records. Always search first.

HubSpot's duplicate management includes a setting to prevent duplicate creation automatically. Salesforce has similar safeguards. If you use a spreadsheet, you need a manual gate — a checklist or spreadsheet column that must be checked before a new row is added.

How to spot new duplicates early

Run a duplicate report monthly. Export your contacts, run the COUNTIF formula again, and check for new duplicates. If the count is increasing, your prevention rules are not working. Investigate the source. Is it a specific form? A specific import? A specific person on your team?

Most CRMs let you save dedup reports as scheduled jobs. Run them monthly and flag any increase as a sign that prevention needs to be reinforced.

The cost of not doing it

Duplicate records compound over time. After a year, what started as a 5 percent duplication rate becomes 15 percent. Your lead scoring breaks. Your workflows misfire. Your sales reps waste time wondering why they are seeing the same person seven ways. Your AI models learn from poisoned data.

A one-time cleanup session every 18 months costs a few hours. The cost of running on bad data is constant.

One person is one record. Everything else is noise.

← Back to all articles