I'm trying to use sed to replace template strings in files of the form %XXX% with the value of a variable called XXX in my shell script.
我正在尝试使用sed将%XXX%形式的文件中的模板字符串替换为我的shell脚本中名为XXX的变量的值。
e.g. The following works perfectly
例如以下工作完美
sed "s/%user_home%/$user_home/gi"
So if user_home=fred the following,
所以如果user_home = fred以下,
NameVirtualHost *:80
<VirtualHost *:80>
ServerName %server_name%
ErrorLog /var/log/apache2/%user_home%_webapp_error.log
CustomLog /var/log/apache2/%user_home%_webapp.log common
DocumentRoot /home/%user_home%/web_app/public
</VirtualHost>
becomes,
NameVirtualHost *:80
<VirtualHost *:80>
ServerName %server_name%
ErrorLog /var/log/apache2/fred_webapp_error.log
CustomLog /var/log/apache2/fred_webapp.log common
DocumentRoot /home/fred/web_app/public
</VirtualHost>
The problem is that I want to run the sed command without explicitly knowing the template strings and their variables up front. That is, it looks for %XXX% and then replaces that with the contents of $XXX without caring what the actual name of the variable is.
问题是我想在没有事先明确知道模板字符串及其变量的情况下运行sed命令。也就是说,它会查找%XXX%,然后用$ XXX的内容替换它,而不关心变量的实际名称是什么。
I know its got something to do with back-references but I can't figure out how to use the content of a back-reference as the variable name.
我知道它与反向引用有关但我无法弄清楚如何使用反向引用的内容作为变量名。
I tried,
sed "s/%\([a-z_]\)%/$(\1)/gi"
but this failed to work because it seems to be a looking for a variable called $\1.
但这没有奏效,因为它似乎是在寻找一个名为$ \ 1的变量。
4 个解决方案
#1
3
The problem here is that by the time the sed command is actually run (and therefore by the time it retrieves the variable-name), the sed command must have been fully assembled (including substituting the Bash variable's value into the replacement string); so everything happens in the wrong order.
这里的问题是,当sed命令实际运行时(因此当它检索变量名时),sed命令必须已经完全组装(包括将Bash变量的值替换为替换字符串);所以一切都以错误的顺序发生。
Or, taking a higher-level view, the problem is that sed doesn't know about Bash variables, so you need Bash to provide the details of the variables, but Bash doesn't know about sed replacements, so it doesn't have any way of knowing what variables you need the details of.
或者,采取更高级别的观点,问题是sed不知道Bash变量,所以你需要Bash来提供变量的细节,但是Bash不知道sed替换,所以它没有任何方式知道你需要什么变量的细节。
The fix, as long as you want to use Bash variables, is to use more Bash: you need to identify the relevant variable-names before you first call sed. The below shows how you can do that.
只要你想使用Bash变量,修复就是使用更多的Bash:你需要在第一次调用sed之前识别相关的变量名。以下显示了如何做到这一点。
To get the list of all variable-names in your file, you can write something like this:
要获取文件中所有变量名的列表,可以编写如下内容:
grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u
(The first grep gets all expressions of the form %...%. The second grep filters out the percent-signs; or you can use sed for that, if you prefer. The sort -u eliminates the duplicates, since you only need the list of distinct variable-names.)
(第一个grep获取%...%形式的所有表达式。第二个grep过滤掉百分号;或者你可以使用sed,如果你愿意的话。排序-u消除重复,因为你只需要不同变量名列表。)
Armed with that, you can assemble a sed command that performs all the necessary replacements:
有了这个,您可以组装一个执行所有必要替换的sed命令:
sed_args=()
while read varname ; do
sed_args+=(-e "s/%$varname%/${!varname}/g")
done < <(grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u)
sed "${sed_args[@]}" FILE
(Note the use of ${!varname} to mean "take the value of $varname as a variable-name, and return the value of that variable." This is what §3.5.3 "Shell Parameter Expansion" of the Bash Reference Manual calls "indirect expansion".)
(注意使用$ {!varname}来表示“将$ varname的值作为变量名,并返回该变量的值。”这就是Bash参考的§3.5.3“Shell参数扩展”手动调用“间接扩展”。)
You can wrap this in a function:
你可以将它包装在一个函数中:
function replace_bash_variables () {
local file="$1"
local sed_args=()
local varname
while read varname ; do
sed_args+=(-e "s/%$varname%/${!varname}/g")
done < <(grep -o '%[a-z_][a-z_]*%' "$file" | grep -o '[a-z_][a-z_]*' | sort -u)
if [[ "${#sed_args[@]}" = 0 ]] ; then
# if no variables to replace, just cat the file:
cat -- "$file"
else
sed "${sed_args[@]}" -- "$file"
fi
}
replace_bash_variables OLD_FILE > NEW_FILE
You can also adjust the above to do line-by-line processing, so that it doesn't need to read the file twice. (That gives you more flexibility, since reading the file twice means you have to pass in the actual file, and can't (say) apply this to the output of a pipeline.)
您还可以调整上述内容以进行逐行处理,这样就不需要两次读取文件。 (这为您提供了更大的灵活性,因为读取文件两次意味着您必须传入实际文件,并且不能(比方说)将此应用于管道的输出。)
#2
0
Use this:
sed -E "s/%(\w+)%/\$\1/g"
For example this:
例如:
echo "abcdef %variable% blah" | sed -E "s/%(\w+)%/\$\1/g"
prints:
abcdef $variable blah
#3
0
Using awk you can do this
使用awk你可以做到这一点
awk '{gsub(/%user_home%/,"${user_home}")}1' file
NameVirtualHost *:80
<VirtualHost *:80>
ServerName %server_name%
ErrorLog /var/log/apache2/${user_home}_webapp_error.log
CustomLog /var/log/apache2/${user_home}_webapp.log common
DocumentRoot /home/${user_home}/web_app/public
</VirtualHost>
This replace the %user_home% to the variable ${user_home}
这将%user_home%替换为变量$ {user_home}
#4
0
try with 1 sed but still need previously to catch the "set" content to know variables name and value
尝试使用1 sed但仍需要先捕获“set”内容以了解变量名称和值
#!/bin/ksh
# YourFilename contain the file name of your file to treat (here passed as 1st parameter to a script)
YourFileName=$1
(set | sed 's/.*/#V0r:&:r0V#/'; cat ${YourFileName}) | sed -n "
s/$/²/
H
$ {
x
s/^\(\n *\)*//
# also reset t flag
t varxs
:varxs
s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\(\n.*\)%\1%/#V0r:\1=\2:r0V#²\3\2/
t varxs
: tmpb
# clean the line when no more occurance in text
# s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\n//
s/^[^²]*:r0V#²\n//
# and next
t varxs
# clean the marker
s/²\(\n\)/\1/g
s/²$//
# display the result
p
}
"
- limitation here due to the use of char "²" not escaped so if ² appear in the file, could be annoying (so change this char as marker or translate it in the file)
- #V0r: and :r0V# are marker also and could be changed without problem
限制在这里由于使用char“²”没有转义所以如果²出现在文件中,可能很烦人(所以将此char更改为标记或将其翻译在文件中)
#V0r:和:r0V#也是标记,可以毫无问题地更改
#1
3
The problem here is that by the time the sed command is actually run (and therefore by the time it retrieves the variable-name), the sed command must have been fully assembled (including substituting the Bash variable's value into the replacement string); so everything happens in the wrong order.
这里的问题是,当sed命令实际运行时(因此当它检索变量名时),sed命令必须已经完全组装(包括将Bash变量的值替换为替换字符串);所以一切都以错误的顺序发生。
Or, taking a higher-level view, the problem is that sed doesn't know about Bash variables, so you need Bash to provide the details of the variables, but Bash doesn't know about sed replacements, so it doesn't have any way of knowing what variables you need the details of.
或者,采取更高级别的观点,问题是sed不知道Bash变量,所以你需要Bash来提供变量的细节,但是Bash不知道sed替换,所以它没有任何方式知道你需要什么变量的细节。
The fix, as long as you want to use Bash variables, is to use more Bash: you need to identify the relevant variable-names before you first call sed. The below shows how you can do that.
只要你想使用Bash变量,修复就是使用更多的Bash:你需要在第一次调用sed之前识别相关的变量名。以下显示了如何做到这一点。
To get the list of all variable-names in your file, you can write something like this:
要获取文件中所有变量名的列表,可以编写如下内容:
grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u
(The first grep gets all expressions of the form %...%. The second grep filters out the percent-signs; or you can use sed for that, if you prefer. The sort -u eliminates the duplicates, since you only need the list of distinct variable-names.)
(第一个grep获取%...%形式的所有表达式。第二个grep过滤掉百分号;或者你可以使用sed,如果你愿意的话。排序-u消除重复,因为你只需要不同变量名列表。)
Armed with that, you can assemble a sed command that performs all the necessary replacements:
有了这个,您可以组装一个执行所有必要替换的sed命令:
sed_args=()
while read varname ; do
sed_args+=(-e "s/%$varname%/${!varname}/g")
done < <(grep -o '%[a-z_][a-z_]*%' FILE | grep -o '[a-z_][a-z_]*' | sort -u)
sed "${sed_args[@]}" FILE
(Note the use of ${!varname} to mean "take the value of $varname as a variable-name, and return the value of that variable." This is what §3.5.3 "Shell Parameter Expansion" of the Bash Reference Manual calls "indirect expansion".)
(注意使用$ {!varname}来表示“将$ varname的值作为变量名,并返回该变量的值。”这就是Bash参考的§3.5.3“Shell参数扩展”手动调用“间接扩展”。)
You can wrap this in a function:
你可以将它包装在一个函数中:
function replace_bash_variables () {
local file="$1"
local sed_args=()
local varname
while read varname ; do
sed_args+=(-e "s/%$varname%/${!varname}/g")
done < <(grep -o '%[a-z_][a-z_]*%' "$file" | grep -o '[a-z_][a-z_]*' | sort -u)
if [[ "${#sed_args[@]}" = 0 ]] ; then
# if no variables to replace, just cat the file:
cat -- "$file"
else
sed "${sed_args[@]}" -- "$file"
fi
}
replace_bash_variables OLD_FILE > NEW_FILE
You can also adjust the above to do line-by-line processing, so that it doesn't need to read the file twice. (That gives you more flexibility, since reading the file twice means you have to pass in the actual file, and can't (say) apply this to the output of a pipeline.)
您还可以调整上述内容以进行逐行处理,这样就不需要两次读取文件。 (这为您提供了更大的灵活性,因为读取文件两次意味着您必须传入实际文件,并且不能(比方说)将此应用于管道的输出。)
#2
0
Use this:
sed -E "s/%(\w+)%/\$\1/g"
For example this:
例如:
echo "abcdef %variable% blah" | sed -E "s/%(\w+)%/\$\1/g"
prints:
abcdef $variable blah
#3
0
Using awk you can do this
使用awk你可以做到这一点
awk '{gsub(/%user_home%/,"${user_home}")}1' file
NameVirtualHost *:80
<VirtualHost *:80>
ServerName %server_name%
ErrorLog /var/log/apache2/${user_home}_webapp_error.log
CustomLog /var/log/apache2/${user_home}_webapp.log common
DocumentRoot /home/${user_home}/web_app/public
</VirtualHost>
This replace the %user_home% to the variable ${user_home}
这将%user_home%替换为变量$ {user_home}
#4
0
try with 1 sed but still need previously to catch the "set" content to know variables name and value
尝试使用1 sed但仍需要先捕获“set”内容以了解变量名称和值
#!/bin/ksh
# YourFilename contain the file name of your file to treat (here passed as 1st parameter to a script)
YourFileName=$1
(set | sed 's/.*/#V0r:&:r0V#/'; cat ${YourFileName}) | sed -n "
s/$/²/
H
$ {
x
s/^\(\n *\)*//
# also reset t flag
t varxs
:varxs
s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\(\n.*\)%\1%/#V0r:\1=\2:r0V#²\3\2/
t varxs
: tmpb
# clean the line when no more occurance in text
# s/^#V0r:\([a-zA-Z0-9_]\{1,\}\)=\([^²]*\):r0V#²\n//
s/^[^²]*:r0V#²\n//
# and next
t varxs
# clean the marker
s/²\(\n\)/\1/g
s/²$//
# display the result
p
}
"
- limitation here due to the use of char "²" not escaped so if ² appear in the file, could be annoying (so change this char as marker or translate it in the file)
- #V0r: and :r0V# are marker also and could be changed without problem
限制在这里由于使用char“²”没有转义所以如果²出现在文件中,可能很烦人(所以将此char更改为标记或将其翻译在文件中)
#V0r:和:r0V#也是标记,可以毫无问题地更改