All Over Tools logo
Tool Combiner
Analyze Tools Clean Tools Compare Tools Convert Tools Data Validation Extract Tools Format Tools Generate Tools Organize Tools Transform Tools
Image Tools Video Tools Audio Tools Subtitle Tools Font Tools
Analyze Tools Clean Tools Compress Tools Convert Tools Edit Tools Extract Tools Generate Tools Merge Tools Organize Tools Redact Tools Repair Tools Split Tools
Analyze Tools Clean Tools Compare Tools Convert Tools Extract Tools Generate Tools Merge Tools Transform Tools Validate Tools
Business Calculators Construction Calculators Date And Time Calculators Education Calculators Financial Calculators Food Calculators Health Calculators Math Calculators Science Calculators Sports Calculators Statistics Calculators
Blog About Us Contact Us

Fuzzy Duplicate Finder

Upload a CSV, TSV, JSON, JSONL, or spreadsheet file and find likely duplicate records using configurable text similarity across the columns you choose. Candidates are grouped for your review - you decide which record to keep or how to merge them. Nothing is deleted automatically.

Drop a CSV, TSV, JSON, JSONL, or spreadsheet file here

or

CSV, TSV, JSON, JSONL, XLSX, XLS

How to find and review similar rows

Upload your data and run a similarity match to find candidate duplicate groups. The tool does not automatically delete records. Review each group and choose whether to resolve it or mark it as Not a duplicate.

Results can show Awaiting review, Resolved, or Not a duplicate. Similar rows can be grouped through a chain of matches, so review the full group rather than only one pair of rows.

Choose columns and matching settings

After loading a file, all discovered columns are selected initially. Use Select all or Select none, then choose the fields that should affect matching. Keep at least one column selected before running the scan.

Set Similarity method to Combined (Levenshtein + token overlap), Levenshtein (character-level, good for typos), or Token Jaccard (word-level, good for reordered text). The Similarity threshold starts at 80% and can be set from 40% to 99%.

Enable Case sensitive when letter case should affect the comparison. Otherwise, values are compared without case differences after surrounding whitespace is removed and repeated whitespace is reduced. Values are converted to text before comparison.

Upload a supported file

Drop a file into the upload area or click browse files. The page accepts CSV, TSV, JSON, JSONL, XLSX, and XLS files.

  • A JSON file must contain an array of records or an object containing a nested array of objects.
  • For spreadsheet files, only the first worksheet is read.
  • The page warns when a file has more than 1,500 rows. Files above the 6,000-row limit are rejected.

Review groups and resolve duplicates

  1. Choose the comparison columns and settings, then click Find duplicate candidates.
  2. In each candidate group, select the base row with its Keep radio button.
  3. Optionally choose a value from another record for each field in the merge controls. These choices start with the selected Keep record.
  4. Click Keep selected, remove rest to resolve the group, or click Not a duplicate to leave it unchanged.
  5. Use Undo / re-review to reopen a resolved or dismissed group.

Export the results

Click Export resolved dataset (CSV) to download the current dataset. You can export while groups are still marked Awaiting review; after confirmation, unresolved groups remain unchanged.

Click Export group report (JSON) to download a report containing each group's status, similarity, record data, and field merge-source information.

Limits and input troubleshooting

Matching may take longer for larger files. If a file exceeds 1,500 rows, the page displays a warning. A file above 6,000 rows is not loaded for matching. If JSON parsing fails, check that the file contains the required array structure. For spreadsheets, move the data you want to read to the first worksheet.

Frequently Asked Questions

How is this different from a Remove Duplicate Rows tool?

A Remove Duplicate Rows tool deletes rows that are exactly identical. This tool finds rows that are similar and groups them for review. It does not delete or merge anything until you make an explicit decision for a group.

How is similarity calculated?

For each selected column, the tool uses the method chosen in Similarity method: Levenshtein, token Jaccard, or the combined average of both. The per-column scores are averaged into an overall similarity and compared with your threshold.

Can I merge fields from different records in a group?

Yes. Select a base record with Keep, then use the field controls to choose a value from any record in the group before clicking Keep selected, remove rest.

Does this tool modify or upload my file?

No. Everything runs locally in your browser. Your original file is never rewritten or uploaded; you export a new resolved file only when you choose to.

Related Tools

34 views