I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something like this
我想找一个bash命令,让我grep目录中的每个文件,并将该grep的输出写入一个单独的文件。我的猜测是做这样的事情
ls -1 | xargs -I{} "grep ABC '{}' > '{}'.out"
but, as far as I know, xargs doesn't like the double-quotes. If I remove the double-quotes, however, then the command redirects the output of the entire command to a single file called '{}'.out instead of to a series of individual files.
但是,据我所知,xargs不喜欢双引号。但是,如果我删除双引号,则命令会将整个命令的输出重定向到名为“{}”的单个文件。而不是一系列单个文件。
Does anyone know of a way to do this using xargs? I just used this grep scenario as an example to illustrate my problem with xargs so any solutions that don't use xargs aren't as applicable for me.
有没有人知道使用xargs做到这一点的方法?我只是用这个grep场景作为例子来说明我对xargs的问题,所以任何不使用xargs的解决方案都不适用于我。
3 个解决方案
#1
Do not make the mistake of doing this:
不要犯这样做的错误:
sh -c "grep ABC {} > {}.out"
This will break under a lot of conditions, including funky filenames and is impossible to quote right. Your {} must always be a single completely separate argument to the command to avoid code injection bugs. What you need to do, is this:
这将在很多条件下破解,包括时髦的文件名,并且无法正确引用。您的{}必须始终是命令的一个完全独立的参数,以避免代码注入错误。你需要做的是这样的:
xargs -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}
Applies to xargs as well as find.
适用于xargs以及查找。
By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).
顺便说一句,永远不要使用没有-0选项的xargs(除非非常罕见和受控的一次性交互使用,你不担心会破坏你的数据)。
Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs
也不要解析ls。永远。使用通配或查找:http://mywiki.wooledge.org/ParsingLs
Use find for everything that needs recursion and a simple loop with a glob for everything else:
将find用于需要递归的所有内容,并使用一个带有glob的简单循环来实现其他所有内容:
find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;
or non-recursive:
for file in *; do grep "$file" > "$file.out"; done
Notice the proper use of quotes.
注意正确使用引号。
#2
A solution without xargs is the following:
没有xargs的解决方案如下:
find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;
...and the same can be done with xargs, it turns out:
......并且xargs也可以这样做,事实证明:
ls -1 | xargs -I {} sh -c "grep ABC '{}' > '{}.out'"
Edit: single quotes added after remark by lhunath.
编辑:lhunath发表评论后添加单引号。
#3
I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:
我假设你的例子只是一个例子,你可能需要>其他的东西。 GNU Parallel http://www.gnu.org/software/parallel/可能是你的救援。只要您的文件名不包含\ n,它就不需要额外的引用:
ls | parallel "grep ABC {} > {}.out"
If you have filenames with \n in it:
如果您的文件名包含\ n:
find . -print0 | parallel -0 "grep ABC {} > {}.out"
As an added bonus you get the jobs run in parallel.
作为额外的奖励,您可以并行运行作业。
Edit.
You can install GNU Parallel simply by:
您可以通过以下方式安装GNU Parallel:
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
Watch the intro videos to learn more: http://pi.dk/1
观看介绍视频以了解更多信息:http://pi.dk/1
10 seconds installation:
10秒安装:
wget pi.dk/3 -qO - | sh -x
#1
Do not make the mistake of doing this:
不要犯这样做的错误:
sh -c "grep ABC {} > {}.out"
This will break under a lot of conditions, including funky filenames and is impossible to quote right. Your {} must always be a single completely separate argument to the command to avoid code injection bugs. What you need to do, is this:
这将在很多条件下破解,包括时髦的文件名,并且无法正确引用。您的{}必须始终是命令的一个完全独立的参数,以避免代码注入错误。你需要做的是这样的:
xargs -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}
Applies to xargs as well as find.
适用于xargs以及查找。
By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).
顺便说一句,永远不要使用没有-0选项的xargs(除非非常罕见和受控的一次性交互使用,你不担心会破坏你的数据)。
Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs
也不要解析ls。永远。使用通配或查找:http://mywiki.wooledge.org/ParsingLs
Use find for everything that needs recursion and a simple loop with a glob for everything else:
将find用于需要递归的所有内容,并使用一个带有glob的简单循环来实现其他所有内容:
find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;
or non-recursive:
for file in *; do grep "$file" > "$file.out"; done
Notice the proper use of quotes.
注意正确使用引号。
#2
A solution without xargs is the following:
没有xargs的解决方案如下:
find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;
...and the same can be done with xargs, it turns out:
......并且xargs也可以这样做,事实证明:
ls -1 | xargs -I {} sh -c "grep ABC '{}' > '{}.out'"
Edit: single quotes added after remark by lhunath.
编辑:lhunath发表评论后添加单引号。
#3
I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:
我假设你的例子只是一个例子,你可能需要>其他的东西。 GNU Parallel http://www.gnu.org/software/parallel/可能是你的救援。只要您的文件名不包含\ n,它就不需要额外的引用:
ls | parallel "grep ABC {} > {}.out"
If you have filenames with \n in it:
如果您的文件名包含\ n:
find . -print0 | parallel -0 "grep ABC {} > {}.out"
As an added bonus you get the jobs run in parallel.
作为额外的奖励,您可以并行运行作业。
Edit.
You can install GNU Parallel simply by:
您可以通过以下方式安装GNU Parallel:
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
Watch the intro videos to learn more: http://pi.dk/1
观看介绍视频以了解更多信息:http://pi.dk/1
10 seconds installation:
10秒安装:
wget pi.dk/3 -qO - | sh -x