如何在复杂的文本文件中替换shell变量

I have several text files in which I have introduced shell variables ($VAR1 or $VAR2 for instance).

我有几个文本文件，其中介绍了shell变量(例如$VAR1或$VAR2)。

I would like to take those files (one by one) and save them in new files where all variables would have been replaced.

我想把这些文件(一个一个地)保存到新的文件中，在那里所有的变量都会被替换。

To do this, I used the following shell script (found on *):

为此，我使用了以下shell脚本(在*上可以找到):

while read line
do
    eval echo "$line" >> destination.txt
done < "source.txt"

This works very well on very basic files.

这在非常基本的文件上工作得非常好。

But on more complex files, the "eval" command does too much:

但是对于更复杂的文件，“eval”命令做了太多:

• Lines starting with "#" are skipped

  以“#”开头的行被跳过

• XML files parsing results in tons of errors

  XML文件解析会导致大量错误

Is there a better way to do it? (in shell script... I know this is easily done with Ant for instance)

有更好的方法吗?(在shell脚本…我知道这很容易用Ant完成)

Kind regards

亲切的问候

8 个解决方案

#1

Looking, it turns out on my system there is an envsubst command which is part of the gettext-base package.

在我的系统上有一个envsubst命令，它是gettext-base包的一部分。

So, this makes it easy:

所以，这很简单:

envsubst < "source.txt" > "destination.txt"

#2

In reference to answer 2, when discussing envsubst, you asked "How can I make it work with the variables that are declared in my .sh script?"

关于回答2，当讨论envsubst时，您问“如何使它与.sh脚本中声明的变量一起工作?”

The answer is you simply need to export your variables before calling envsubst.

答案是您只需在调用envsubst之前导出变量即可。

You can also limit the variable strings you want to replace in the input using the envsubst SHELL_FORMAT argument (avoiding the unintended replacement of a string in the input with a common shell variable value - e.g. $HOME).

您还可以使用envsubst SHELL_FORMAT参数限制您想要在输入中替换的变量字符串(避免在输入中意外地使用公共shell变量值替换字符串——例如$HOME)。

For instance:

例如:

export VAR1='somevalue' VAR2='someothervalue'
MYVARS='$VAR1:$VAR2'

envsubst "$MYVARS" <source.txt >destination.txt

Will replace all instances of $VAR1 and $VAR2 (and only VAR1 and VAR2) in source.txt with 'somevalue' and 'someothervalue' respectively.

将在源代码中替换$VAR1和$VAR2的所有实例(仅替换VAR1和VAR2)。txt与“somevalue”和“someothervalue”分别对应。

#3

I know this topic is old, but I have a simpler working solution without export the variables. Can be a oneliner, but I prefer to split using \ on line end.

我知道这个主题已经过时了，但是我有一个更简单的工作解决方案，不导出变量。可以是一个oneliner，但我更喜欢使用线端\。

var1='myVar1'\
var2=2\
var3=${var1}\
envsubst '$var1,$var3' < "source.txt" > "destination.txt"

#         ^^^^^^^^^^^    ^^^^^^^^^^     ^^^^^^^^^^^^^^^
# define which to replace   input            output

The variables need to be defined to the same line as envsubst is to get considered as environment variables.

需要将变量定义为与envsubst相同的行，将其视为环境变量。

The '$var1,$var3' is optional to only replace the specified ones. Imagine an input file containing ${VARIABLE_USED_BY_JENKINS} which should not be replaced.

“$var1，$var3”是可选的，只能替换指定的变量。想象一个包含${VARIABLE_USED_BY_JENKINS}的输入文件，它不应该被替换。

#4

1. Define your ENV variable
2. 定义你的ENV变量

$ export MY_ENV_VAR=congratulation

2. Create template file (in.txt) with following content
3. 创建包含以下内容的模板文件(in.txt)

$MY_ENV_VAR

You can also use all other ENV variables defined by your system like (in linux) $TERM, $SHELL, $HOME...

您还可以使用系统定义的所有其他ENV变量，比如(在linux中)$TERM、$SHELL、$HOME…

3. Run this command to raplace all env-variables in your in.txt file and to write the result to out.txt
4. 运行这个命令，让所有的env变量在你的里面。并将结果写入out.txt文件

$ envsubst "`printf '${%s} ' $(sh -c "env|cut -d'=' -f1")`" < in.txt > out.txt

4. Check the content of out.txt file
5. 检查输出内容。txt文件

$ cat out.txt

and you should see "congratulation".

你应该看到"恭喜"。

#5

If you really only want to use bash (and sed), then I would go through each of your environment variables (as returned by set in posix mode) and build a bunch of -e 'regex' for sed from that, terminated by a -e 's/\$[a-zA-Z_][a-zA-Z0-9_]*//g', then pass all that to sed.

如果您真的只想使用bash(和sed)，那么我将遍历您的每个环境变量(通过posix模式的set返回)，并为sed构建一组-e 'regex，以-e 's/\$[a- za - z_][a- za - z0 -9_]*/ g'结束，然后将所有这些传递给sed。

Perl would do a nicer job though, you have access to the environment vars as an array and you can do executable replacements so you only match any environment variable once.

Perl可以做得更好，您可以作为数组访问环境vars，还可以执行可执行的替换，因此您只能匹配任何环境变量一次。

#6

Actually you need to change your read to read -r which will make it ignore backslashes.

实际上，你需要将你的阅读改为读-r，这将使它忽略反斜杠。

Also, you should escape quotes and backslashes. So

同样，你应该避免引用和反斜杠。所以

while read -r line; do
  line="${line//\\/\\\\}"
  line="${line//\"/\\\"}"
  line="${line//\`/\\\`}"
  eval echo "\"$line\""
done > destination.txt < source.txt

Still a terrible way to do expansion though.

不过，这仍然是一种糟糕的扩张方式。

#7

envsubst seems exactly like something I wanted to use, but -v option surprised me a bit.

envsubst似乎就是我想要使用的东西，但是-v选项让我有点吃惊。

While envsubst < template.txt was working fine, the same with option -v was not working:

虽然envsubst <模板。txt运行良好，选项-v也不工作:< p>

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.1 (Maipo)
$ envsubst -V
envsubst (GNU gettext-runtime) 0.18.2
Copyright (C) 2003-2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Bruno Haible.

As I wrote, this was not working:

正如我所写的，这行不通:

$ envsubst -v < template.txt
envsubst: missing arguments
$ cat template.txt | envsubst -v
envsubst: missing arguments

I had to do this to make it work:

我必须这么做才能成功:

TEXT=`cat template.txt`; envsubst -v "$TEXT"

Maybe it helps someone.

也许它可以帮助别人。

#8

Export all the needed varaibles and then use a perl onliner

导出所需的所有varaibles，然后使用perl onliner

TEXT=$(echo "$TEXT"|perl -wpne 's#\${?(\w+)}?# $ENV{$1} // $& #ge;')

文本= $(echo " $文本”| perl -wpne’s # \ $ { ?(\ w +)} ?# $ENV{$1} / $& #ge;')

This will replace all the ENV variables present in TEXT with actual values. Quotes are also preserved :)

这将用实际值替换文本中出现的所有ENV变量。引号也被保留:)