Removing duplicate lines with uniq

To remove duplicate lines from a file use the command:

   uniq in_file out_file

uniq searches the in_file for consecutive duplicate lines - a line that is immediately followed by one or more lines that are identical to it - and removes them. The result is an out_file that contains no duplicate lines.

Use the sort command if you want to remove duplicate lines that occur in different parts of a file.

[Home] [Search] [Index] This site maintained by webmaster@deslab.mit.edu