I need to remove similar (not identical) lines from a file. e.g.
我需要从文件中删除类似(不相同)的行。例如
file.txt
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html
Wanted results: (unique lines ending with #botas)
想要的结果:(以#botas结尾的独特线条)
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas
Any handy solution?
任何方便的解决方案?
1 个解决方案
#1
2
With awk:
用awk:
awk -F\# '!a[$1]++' your_file.txt
Output:
输出:
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas
#1
2
With awk:
用awk:
awk -F\# '!a[$1]++' your_file.txt
Output:
输出:
/Bota-Espaco-Fashion-Com-Tachas-e-Cristais-Bege-1472878.html#botas
/Bota-Raphaella-Booz-Cano-Curto-2-Fivelas-Vermelha-1458535.html#botas
/Bota-Dumond-Country-3-Fivelas-Caramelo-1481004.html#botas