The uniq is used to remove duplicate lines from a file on Unix-like operating systems. The file needs to be sorted prior.

Syntax

The basic syntax for the uniq command is as follows:

uniq [options] <file>

The <file> argument is the path to the file that you want to remove duplicate lines from. If you do not specify a file, the uniq command will read from STDIN.

Examples

Remove duplicate lines from the file “myfile.txt”.

uniq myfile.txt

Remove duplicate lines from the file “myfile.txt” and ignore case. uniq -i myfile.txt

Remove duplicate lines from the file “myfile.txt” and only print the unique lines.

uniq -u myfile.txt

Options

The uniq command has a few options that can be used to modify its behavior. The most common options are:

-c: Count the number of occurrences of each line.

-d: Print the duplicate lines only.

-i: Ignore case when comparing lines.

-u: Print the unique lines only.

For more information on the uniq command and its options, please see the man page: man uniq.

With gencmd

gencmd sort a file and then show only unique lines

• sort -u
• sort -u file

gencmd -c uniq sort a file and then show only unique lines uniq -c

• $ sort | uniq
• $ sort file | uniq