Prepare the input
Paste one record per line. Run Whitespace Cleaner first when invisible leading spaces, tabs, or inconsistent blank blocks should not distinguish otherwise identical values.
Keep the first occurrence of each line
Duplicate Line Remover
Remove repeated whole lines in original order, with case-sensitive comparison and blank-line policy under your control and no server upload.
- 01Private by designYour text stays in this browser tab.
- 02Bounded and responsiveUp to 500,000 characters in each main text field.
- 03Ready to reuseCopy the result or download a branded TXT file.
Counting and transformations run locally. Nirmion does not send the text in this workspace to an API.
Product guide
Define what a duplicate means before removing it
Whole-line comparison is predictable but intentionally strict. Case-insensitive mode folds letter case only; it does not automatically trim punctuation, normalize spelling, or collapse internal whitespace.Choose the rule
Choose case-insensitive comparison to treat Alpha and alpha as the same key, or sensitive comparison to retain both. Decide whether the first blank line is a legitimate item or all blank lines should be removed.
Review the output
Review retained capitalization and order because the first occurrence wins. For customer, finance, inventory, or compliance records, use stable identifiers and structured duplicate review rather than relying only on display text.
How it works
One forward pass, one retained first occurrence
Nirmion normalizes line endings and walks the list in source order. A Set stores either the exact line or its locale-lowercase form. The first unseen key is retained; later matching keys are counted and removed. Blank lines follow the selected policy. No fuzzy matching, stemming, or similarity model is applied.
Useful when
Clean repeated line values while preserving order
First-occurrence retention works well for lists where sequence matters and every line is independently reviewable.
- Remove repeated tags, URLs, email-domain lists, keywords, checklist items, or copied dropdown values.
- Deduplicate merged notes or exports while retaining the earliest spelling and original order.
- Preflight a list before sorting, importing, comparing, or handing it to a structured validation workflow.
Questions answered
Duplicate Line Remover questions
Practical boundaries matter with text tools. These answers explain what is measured, what is retained, and when a manual review is still needed.
Are leading spaces ignored?
No. Lines are compared as entered, except optional case folding. Use Whitespace Cleaner first if leading, trailing, or repeated horizontal spaces should be normalized.
Which duplicate is kept?
The first occurrence in source order. Its original spelling and spacing are preserved.
Does it find similar or misspelled lines?
No. There is no fuzzy matching. Similar names, punctuation variants, and spelling differences remain separate because automatically merging them could destroy valid data.
Privacy and retention
Your draft stays on your device.
Nirmion performs these operations with browser JavaScript. The workspace does not call the catalogue API with your source text, and completed-job content is kept only in this component memory for re-download.
A downloaded file remains in your browser downloads, device storage, backups, and any location where you share it. Clear or close the page when working on a shared device, and avoid pasting secrets into extensions or third-party clipboard managers.
Tool details
Remove exact repeated lines without reordering the list
Nirmion Duplicate Line Remover keeps the first occurrence of each whole line and removes later matches. Users can choose case-sensitive or case-insensitive comparison and control blank-line retention. Metrics make the number of removed lines explicit before the result is copied or downloaded.
Processing stays local for lists up to 500,000 characters, and output downloads as text_nirmion_tools_deduplicated.txt. The page distinguishes exact line cleanup from fuzzy entity matching so the tool is used safely for ordinary lists rather than critical record linkage.