mastros

Sales Teams: Deduplicate Exported Leads Locally, Don’t Delete Blindly

October 14, 2026·
Sales Teams: Deduplicate Exported Leads Locally, Don’t Delete Blindly

Export your current CRM contacts alongside the new lead file, pick email as the identity key, normalize both, run an exact pass and then a conservative fuzzy pass, and review a duplicates-only report before touching a single row. Then import updates (matched by Record ID) and new contacts as two separate files. Exports from tools like Mastros make the first step easy.

TL;DR:

  • When email is missing, match on phone or name plus company; split cells containing multiple addresses, and include alternate CRM emails to catch overlooked contacts.
  • Use Record IDs to route existing contacts to an update file, keep new contacts separate, and test a small batch before loading the full files.
  • Keep the original export, CRM snapshot, source row references, and merge history together, because HubSpot warns that record merges are irreversible.
  • Process prospect files locally and require a preview of duplicate groups with row references; uploading them to an unvetted web tool exposes personal data.

Table of Contents

Quick dedupe checklist you can run now on a CSV or Excel export

Good news: the core routine becomes quicker once you have done it before. Work through these steps in order.

  1. Save an untouched original copy, then add export metadata to the filename or a header row: source platform and export date.
  2. Choose your identity keys. Email is the primary key. Fall back to phone, or to name plus company, for rows with no email. Use more than one key when a single column is often blank.
  3. Normalize every key column: trim whitespace, lowercase text, strip stray punctuation, and split multi-value cells (two emails in one field) into separate columns.
  4. Run an exact-match pass on email first. Then run a fuzzy pass to catch near-misses like "Jon Smith" and "Jonathan Smith" at the same company.
  5. Annotate each row with dup_group and dup_count rather than deleting anything.
  6. Output three files: a duplicates-only file for review, the annotated full file, and the deduped file you intend to import.

Pro Tip: Add a source_row column before you sort anything, so every flagged duplicate can be traced back to its original line in the raw export.

Step-by-step methods: spreadsheet formulas, fuzzy matching, and csv tools

You do not need special software to flag duplicates. Pick the method that matches your file size and comfort level.

  • Exact match in Excel or Google Sheets: =COUNTIF(A:A, A2)>1 flags any email that appears more than once. UNIQUE Returns a clean list of distinct values, and FILTER can pull out only the rows where the count exceeds one. For combined keys, build a helper column such as =LOWER(TRIM(B2))&"|"&LOWER(TRIM(C2)) and run COUNTIF on that.
  • Matching against your CRM export: XLOOKUP (or VLOOKUP in older Excel) looks up each new email in the exported CRM column and returns the existing Record ID. A hit means "update"; an error means "new."
  • Fuzzy matching: approximate matching compares names by tokens, partial strings, or edit distance. Fuzzy add-ons exist for both Excel and Sheets. Keep the threshold conservative, because a loose setting will happily pair two different people who share a common surname at the same company.
  • Command line: csvkit-style tooling can group duplicates and write a duplicates-only report annotated with dup_group, dup_count, and source_row, with separate output modes for the report, the annotated full file, and the deduped file. That is the "report, don't delete" principle in practice.
  • Keep-first, keep-last, or keep-most-complete: for keep-first, sort by import order and keep rows where the group counter equals one. For keep-last, sort by a timestamp descending. For most-complete, add a helper column that counts non-empty fields per row (=COUNTA(A2:H2)) and keep the highest score in each group.

Pro Tip: Break ties on the most recent activity date, so the surviving row is both the fullest and the freshest.

CRM-aware workflow: use Record ID or email to separate updates from new imports

A CSV deduper cannot see what already lives inside your CRM. That is why we always start by exporting existing contacts with Record ID, primary email, alternate emails, and last activity date. Keep that export as an audit snapshot. HubSpot allows multiple email addresses per contact, so include every address column in the export or you will miss matches.

Then compare the two files:

  • If your CRM accepts Record IDs on import, add the Record ID to matched rows so they update the existing record instead of creating a new one. An XLOOKUP against the CRM export can fill the column.
  • When no Record ID is available, match on email. HubSpot treats email as the default identifier for contacts and domain for companies, while Salesforce lets admins define custom matching and duplicate rules, as HubSpot's duplicate documentation explains.
  • Build two import files: one with Record IDs for updates and one with brand-new contacts.
  • Test a small batch first, then check the audit logs and merge history before loading the rest.

Salesforce matching rules pair with duplicate rules to flag or block duplicate creation, and admins can build reports to track how well those rules perform.

If your leads come from LinkedIn, our guide on exporting LinkedIn to CRM covers formats that keep the fields you need for matching.

Tools and privacy: process locally and preview before anything is removed

Exported prospect lists are personal data. Uploading them to a random web deduper means handing names, emails, and phone numbers to a third party you have not vetted. Local or browser-based processing keeps the raw file on your machine, which lowers your data-sharing and compliance exposure and removes one more vendor from your review process.

Privacy is only half the question. A tool should also show you what it intends to do before it does it. When you evaluate options, check for:

  • A preview mode that lists duplicates instead of silently deleting them
  • Configurable key columns, so you choose email, phone, or a combined key
  • A duplicates-only report with dup_group, dup_count, and original row references
  • Annotated file export, so your review survives outside the tool
  • No forced upload of the file
  • A log or audit export you can attach to your import ticket

The csvkit-style report workflow above meets all of these. If a tool can only hand you a finished "clean" file, you cannot verify what disappeared.

Verification and rollback: produce audit trails and test imports first

Merges are the step you cannot easily undo. HubSpot warns that merging records is irreversible, so the surviving record should be the one with the most complete and most recent activity history. Treat verification as part of the job, not an afterthought.

  1. Generate the duplicates-only report with source_row and dup_group, and read through every flagged group, especially fuzzy matches.
  2. Export or record merge history wherever your CRM supports it, and save the pre-import CRM snapshot next to it.
  3. Import a small sample into a sandbox or a narrow segment, then review the resulting activity and audit logs.
  4. Only after the sample looks right, run the full update file, then the new-contacts file.

Pro Tip: Keep the original export, the report, and the CRM snapshot in one dated folder. If a customer asks why their history looks odd, you can explain what happened.

For very large lists, blocking and sliding-window techniques, like those in the FDup framework, show how candidate pairs get narrowed before detailed matching.

A report-first pipeline beats a fast delete

Teams usually want speed, and I understand why. But the pipeline that lasts is the boring one: a quick exact-match pass, a conservative fuzzy review by a human, and a small test import before the full load. A false merge can cost far more than a duplicate you missed. Reproducible exports with metadata and local, browser-first processing keep privacy-conscious teams moving without anxiety.

— Elias Mahdavi

Get CRM-ready exports without uploading your data

We build privacy-first Chrome extensions that export contacts and search results from LinkedIn, WhatsApp, and Telegram straight into CSV, Excel, or JSON, entirely in your browser. Nothing goes to our servers, so you can run the dedupe checklist above on a clean local file. Once your lists are tidy, a solid lead nurturing strategy tells you what to do with them. Try our LinkedIn Scraper and Sales Navigator Export or see plans on Mastros.

Get CRM-ready exports without uploading your data — overview diagram

FAQ

What is the safest way to deduplicate exported leads?

Flag duplicates instead of deleting them. Produce a duplicates-only report, review it, and only then build the file you import. That keeps every decision traceable to its original row.

Is matching on email alone enough to remove duplicate leads?

Not usually. Exact email matching misses people with several addresses or blank email fields, so add a second pass on phone or name plus company, with a conservative fuzzy threshold.

How do I avoid creating duplicates when importing into HubSpot?

Export your existing contacts with Record ID and every email column, then fill in Record IDs for matched rows using a lookup. HubSpot uses email as the default unique identifier for contacts, so keep emails normalized.

Which record should survive when two contacts are merged?

Keep the one with the most complete and most recent activity, because merges can be irreversible. Check the details before you confirm.

Sources

Recommended

More from the blog