Bash - 如何删除类似(不相同)的行? grep的?桑达? AWK?

时间:2021-05-26 01:02:23

I need to remove similar (not identical) lines from a file. e.g.

我需要从文件中删除类似(不相同)的行。例如

file.txt

/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html

Wanted results: (unique lines ending with #botas)

想要的结果:(以#botas结尾的独特线条)

/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas

Any handy solution?

任何方便的解决方案?

1 个解决方案

#1


2  

With awk:

用awk:

awk -F\# '!a[$1]++' your_file.txt

Output:

输出:

/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas

#1


2  

With awk:

用awk:

awk -F\# '!a[$1]++' your_file.txt

Output:

输出:

/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas