CSV deduplicator
Paste CSV, tick the columns that identify a row, and the tool removes later (or earlier) rows with the same key. You can ignore letter case and extra spaces, and the header row is always kept.
Runs in your browser: nothing you enter is sent anywhere.
How to use it
- Paste CSV. The column names appear as tick boxes.
- Untick columns that should not count when comparing rows.
- Choose whether to ignore case and spaces, and whether to keep the first or last match.
- Copy the result. The line under it tells you how many rows were removed.
How it works
For each data row the tool builds a key from the ticked columns, after trimming and lowercasing if you asked for that. The first time a key appears the row is remembered; with Keep last, a later row replaces the earlier one. The surviving rows are written out in their original order.
Example
In the sample, rows 2 and 4 share the e-mail grace@example.com apart from capital letters, and rows 1 and 3 share ada@example.com once the trailing space is ignored. With only the email column ticked and First match, rows 3 and 4 are removed and three rows remain.
Tips
- To remove only fully identical lines in plain text, the remove duplicate lines tool is simpler.
- Run the CSV viewer first if you are not sure which columns identify a record.
Frequently asked questions
How does the tool decide that two rows are duplicates?
It joins the values of the ticked columns into a key and compares keys. Rows with equal keys are duplicates. With all columns ticked, only rows identical in every column count; tick just the e-mail column to find people who appear twice under different names.
Which row is kept?
The first match by default, so the original order of survivors is preserved. Choose Last match to keep the most recent row of each group; the kept rows still appear in their original order.
Does it change my other data?
No. Kept rows are written back as they were read, with fields quoted only when they contain the delimiter, a double quote or a line break. The header row is never removed.
What do the ignore options do?
Ignoring case treats ADA@x.com and ada@x.com as the same. Ignoring spaces trims the start and end of each compared value; the kept row still has its original text.
Is anything uploaded?
No. The comparison runs in your browser.