[Python正则表达式] 字符串中xml标签的匹配

时间：2023-09-10 22:31:57
[Python正则表达式] 字符串中xml标签的匹配

　　现在有一个需求，比如给定如下数据：

0-0-0 0:0:0 #### the 68th annual golden globe awards ####  the king s speech earns 7 nominations  ####  <LOCATION>LOS ANGELES</LOCATION> <ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION> historical drama British king stammer beat competitors Tuesday grab seven nominations Golden Globe Awards nominations included best film drama nod contested award organizers said films competing best picture <ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION> earned nominations best performance actor olin <PERSON>Firth</PERSON> best performance actress <PERSON>Helena Bonham</PERSON> arter best supporting actor <PERSON>Geoffrey Rush</PERSON> best director <PERSON>Tom Hooper</PERSON> best screenplay <PERSON>David Seidler</PERSON> best movie score <ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION> earned nods apiece Black Swan Inception Kids Right tied place movie race nominations best motion picture comedy musical category <ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION> compete Nominated best actor motion picture olin <ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION> best actress motion picture nominees <PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON> Rabbit Hole <PERSON>Jennifer Lawrence</PERSON> <ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION> categories Glee nominee nods followed Rock Boardwalk Empire Dexter Good Wife Mad Men Modern Family Pillars Earth Temple <PERSON>Grandin</PERSON> tied nods apiece awards announced Jan

　　要求按行把<></>标签内的字符串中的空格替换成下划线_，并且将数据转换形式，例：<X>A B C</X>需要转换成A_B_C/X

　　由于正则表达式匹配是贪婪模式，即尽可能匹配到靠后，那么就非常麻烦，而且仅仅是用?是无法真正保证是非贪婪的。所以需要在正则匹配时给之前匹配好的字符串标一个名字。

python下，正则最终写出来是这样：

 LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')

　　接下来我们需要做是在原字符串中找出对应的子串，并且记下他们的位置，接下来就是预处理出需要替换成的样子，再用一个正则就好了。

 LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

　　对字符串集合做整次的map，对每一个字符串进行匹配，再吧这两部分匹配结果zip在一起，就可以获得一个start-end的tuple，大致这样。

 ('<LOCATION>LOS ANGELES</LOCATION>', 'LOS_ANGELES/LOCATION')

 ('<ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION>', 'Dec_Xinhua_Kings_Speech/ORGANIZATION')

 ('<ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION>', 'Social_Network_Black_Swan_Fighter_Inception_Kings_Speech/ORGANIZATION')

 ('<PERSON>Firth</PERSON>', 'Firth/PERSON')

 ('<PERSON>Helena Bonham</PERSON>', 'Helena_Bonham/PERSON')

 ('<PERSON>Geoffrey Rush</PERSON>', 'Geoffrey_Rush/PERSON')

 ('<PERSON>Tom Hooper</PERSON>', 'Tom_Hooper/PERSON')

 ('<PERSON>David Seidler</PERSON>', 'David_Seidler/PERSON')

 ('<ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION>', 'Alexandre_Desplat_Social_Network_Fighter/ORGANIZATION')

 ('<ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION>', 'Alice_Wonderland_Burlesque_Kids_Right_Red_Tourist/ORGANIZATION')

 ('<ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION>', 'Firth_Kings_Speech_James_Franco_Hours_Ryan_Gosling_Blue_Valentine_Mark_Wahlberg_Fighter_Jesse_Eisenberg_Social_Network/ORGANIZATION')

 ('<PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON>', 'Halle_Berry_Frankie_Alice_Nicole_Kidman/PERSON')

 ('<PERSON>Jennifer Lawrence</PERSON>', 'Jennifer_Lawrence/PERSON')

 ('<ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION>', 'Winters_Bone_Natalie_Portman_Black_Swan_Michelle_Williams_Blue_Valentine_TV/ORGANIZATION')

 ('<PERSON>Grandin</PERSON>', 'Grandin/PERSON')

 ('<LOCATION>BEIJING</LOCATION>', 'BEIJING/LOCATION')

 ('<ORGANIZATION>Xinhua Sanlu Group</ORGANIZATION>', 'Xinhua_Sanlu_Group/ORGANIZATION')

 ('<LOCATION>Gansu</LOCATION>', 'Gansu/LOCATION')

 ('<ORGANIZATION>Sanlu</ORGANIZATION>', 'Sanlu/ORGANIZATION')

　　处理的代码如下：

 def read_file(path):

     if not os.path.exists(path):

         print 'path : \''+ path + '\' not find.'

         return []

     content = ''

     try:

         with open(path, 'r') as fp:

             content += reduce(lambda x,y:x+y, fp)

     finally:

         fp.close()

     return content.split('\n')

 def get_label(each):

     pair = zip(LABEL_PATTERN.findall(each),

                          map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))

     return map(lambda x: (x[0][0], x[1]), pair)

 src = read_file(FILE_PATH)

 pattern = map(get_label, src)

　　接下来简单处理以下就好：

 for i in range(0, len(src)):

     for pat in pattern[i]:

         src[i] = re.sub(pat[0], pat[1], src[i])

　　所有代码：

 # -*- coding: utf-8 -*-

 import re

 import os

 # FILE_PATH = '/home/kirai/workspace/sina_news_process/disworded_sina_news_attr_handled.txt'

 FILE_PATH = '/home/kirai/workspace/sina_news_process/test.txt'

 LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')

 LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

 def read_file(path):

     if not os.path.exists(path):

         print 'path : \''+ path + '\' not find.'

         return []

     content = ''

     try:

         with open(path, 'r') as fp:

             content += reduce(lambda x,y:x+y, fp)

     finally:

         fp.close()

     return content.split('\n')

 def get_label(each):

     pair = zip(LABEL_PATTERN.findall(each),

                          map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))

     return map(lambda x: (x[0][0], x[1]), pair)

 src = read_file(FILE_PATH)

 pattern = map(get_label, src)

 for i in range(0, len(src)):

     for pat in pattern[i]:

         src[i] = re.sub(pat[0], pat[1], src[i])

相关文章