Linux 系统文档行去重工具 uniq

2015-08-13 0 By admin

uniq命令的作用:显示唯一的行,对于那些连续重复的行只显示一次!

一、常用参数

-c, --count 按重复次数给行加前缀。
-u, --unique 显示文件中没有连续出现的行。
-d, --repeated 显示文件中连续重复出现的行。
-i, --ignore-case 比较时忽略差异。
-w, --check-chars= 检查行重复时,只检查前N个字符。
-s, --skip-chars=N 检查行重复时,忽略前N个字符。
-f, --skip-fields= 检查行重复时,忽略前N个字段。

二、使用示例

1、查看演示文本内容
[root@blogs ~]# cat test
boy took bat home
boy took bat home
girl took bat home
dog brought hat home
dog brought hat home
dog brought hat home

2、默认显示连续重复的行一次
[root@blogs ~]# uniq test
boy took bat home
girl took bat home
dog brought hat home

3、显示文件中每行连续出现的次数
[root@blogs ~]# uniq -c test
2 boy took bat home
1 girl took bat home
3 dog brought hat home

4、显示文件中连续重复出现的行
[root@blogs ~]# uniq -d test
boy took bat home
dog brought hat home

5、显示文件中没有连续出现的行
[root@blogs ~]# uniq -u test
girl took bat home

6、比较时忽略行前2个字段
[root@blogs ~]# uniq -f 2 test
boy took bat home

7、比较时忽略行前1个字节
[root@blogs ~]# uniq -s 1 test
boy took bat home
dog brought hat home