Toolsel

Remove Duplicate Lines

Runs in your browser

Strip duplicate lines from a list while keeping the order you choose.

Input

Result

What is a Remove Duplicate Lines?

Deduplicating a list — email addresses exported from two systems, log lines, a list of URLs — comes up constantly, and doing it by eye is error-prone the moment the list passes a few dozen lines. This tool keeps one copy of each distinct line and discards the rest, with control over exactly what "distinct" means for your case.

The two toggles matter more than they look. Case-sensitive comparison treats "Example.com" and "example.com" as different lines; most real-world deduplication (URLs, emails, tags) wants that off. Trimming whitespace catches the case where two lines are identical except for a trailing space that's invisible in a normal text editor — otherwise those show up as "different" for no visible reason.

How to use it

  1. Paste your list, one item per line.
  2. Toggle case-sensitive comparison and whitespace trimming to match what should count as a duplicate.
  3. Choose whether to keep the first or last occurrence of each duplicate.
  4. Optionally sort the result alphabetically.
  5. Copy the deduplicated list — the count of lines removed is shown above the result.

Example

Input
apple
banana
Apple
apple
cherry
Output
apple
banana
cherry

Frequently asked questions

What counts as a duplicate by default?
Exact, case-sensitive matches after trimming leading and trailing whitespace from each line — that combination catches the most common real-world case (a re-pasted list with inconsistent spacing) without silently merging lines that are genuinely different, like differently-cased words that matter in your context.
Does order matter, and can I control it?
By default the first occurrence of each line is kept and the list order is otherwise preserved. Switch to "keep last" if a later entry in your list is the more up-to-date one, or turn on alphabetical sorting if order doesn't matter and you just want a clean, sorted unique list.
Are blank lines treated as duplicates of each other?
Yes — every empty line is identical to every other empty line, so with default settings only the first (or last) blank line survives, the rest are removed along with any other duplicate.