CSV Deduplicator
Remove duplicate CSV records by chosen columns, keep exact values and review removed rows locally.
This tool runs entirely in your browser. Your data is never uploaded, never stored, and never leaves your device.
CSV Deduplicate removes rows sharing exact values in selected key columns, keeps the first or last occurrence, and exports both retained rows and removed duplicates.
How to use it
- 1Paste or load a CSV with unique headers, choose the delimiter and read the columns.
- 2Select duplicate-key columns and whether to keep the first or last source occurrence.
- 3Review the retained and removed rows, then download either CSV.
Example
- Input
- id,name 001,Mira 002,Leo 001,Mira updated Key: id; keep last
- Output
- id,name 002,Leo 001,Mira updated
The earlier 001 row is available in the separate removed-duplicates export.
What happens to your data
Duplicate keys, source rows and both result files are processed in browser memory. CSV contents and selected key values are not uploaded; clearing the editor discards its current result.
Last updated October 2026
Remove duplicate CSV rows while deciding what makes two records the same. Select one key column, several columns together, or every column for exact whole-row deduplication. CSV Deduplicate can keep the first occurrence or the last occurrence and gives you separate downloads for retained rows and removed duplicates.
This is useful when repeated exports overlap but a newer row may contain a corrected name or status. Choosing only an ID treats those differing rows as duplicates; choosing every column preserves them as distinct records. A preview and the duplicate-group count help you review that choice before saving anything.
How it works
Paste a table or load a UTF-8 CSV, choose comma, semicolon or tab, then read the columns. The first record must contain unique, nonempty header names. Every subsequent record must have that many fields. Quoted separators, doubled quotes and quoted line breaks are parsed as field content; malformed quotes or ragged records stop the operation with a record and line location.
Every selected key participates in an exact text comparison. Values are not trimmed, lowercased, parsed as numbers or normalized. The strings 001, 1 and 1.0 are different keys, as are Mira and mira. Empty strings can form a duplicate group just like other values. Composite keys are stored as arrays of strings, so a separator inside a field cannot collide with a neighboring key column.
Keep first retains the earliest source row for each key. Keep last retains the latest source row, including all its non-key cells. Both downloads preserve the relative source order of the rows they contain; keep last does not sort rows by the first appearance of each key. The removed-duplicates CSV contains the original columns, without an extra reason column. If no rows are removed, that download contains the header only. A header-only input produces two header-only exports.
The input limit is 2 MiB, 10,000 records including the header, 200 columns and 200,000 cells. Each CSV output is limited to 10 MiB. Preview cards show up to eight rows and six columns in each result, abbreviating text after 140 characters; downloads contain complete values. Editing the input or delimiter clears the column selection and previous result. Changing key columns, retention or export protection invalidates the prior downloads.
By default, output preserves field text, including long IDs and leading zeros. Optional formula protection prefixes an apostrophe to formula-like exported cells, including headers and negative numbers; it never changes how duplicates are matched. The preview shows original values. This changes the exported strings and is not a universal guarantee for every spreadsheet. Import identifier columns as text when exact spelling matters. Files use UTF-8, CRLF record endings and no BOM; quoted field line breaks remain intact. All matching and export processing happens in the browser.
Worked example: Keep the latest row for each exact ID
- Input
- id,name 001,Mira 002,Leo 001,Mira updated Key: id; keep last; formula protection off
- Result
- id,name 002,Leo 001,Mira updated
The earlier 001 row appears in removed-duplicates.csv. The retained rows remain in their original relative order.
Common use cases
- Removing repeated records after merging overlapping CSV exports
- Keeping the latest source occurrence for a customer or inventory ID
- Removing identical whole rows without merging different records
- Reviewing discarded rows before importing a cleaned file
Frequently asked questions
How do I remove duplicate CSV rows using only one column?＋
Read the columns, leave only that column checked, choose keep first or keep last, and remove duplicates. All other cells come from the selected winning row. The tool does not combine values from different duplicates.
Does keep last mean the newest timestamp?＋
No. It means the last occurrence in the source file. The tool does not parse or sort dates. Sort your source deliberately before using this mode if row order should represent time.
Are spaces, letter case and empty keys ignored?＋
No. Matching is exact. Trailing spaces and letter case matter; an empty key matches another empty key. Selecting an additional column can distinguish rows that share an empty ID.
Can I recover the removed duplicate rows?＋
Download removed-duplicates.csv while the result is present. It contains every discarded row with the source columns. The tool does not retain a history after navigation, reload or clearing the input.
Will long identifiers or formula cells change?＋
Identifiers remain string values in the CSV. Formula-like text is also preserved unless you explicitly enable apostrophe prefixes. A spreadsheet may still infer data types or handle protection differently; use a text-column import workflow when necessary.
Can I remove duplicates from a CSV with no header?＋
This tool needs unique named columns for key selection. Add an intentional header first. The separate CSV Split and CSV Transpose tools support a first record treated as data.
Standards and limitations
Related tools
CSV Merger
Combine CSV tables by header name, preserve IDs and download the result locally.
CSV Filter
Filter CSV rows, select output columns and download a focused table without uploads.
CSV Compare by Key
Compare CSV exports by a unique key. Preserve string IDs and inspect added, removed and changed records.
CSV Validator
Check CSV syntax, row widths and headings without silently repairing data. Download a diagnostic report.