[Python正则表达式] 字符串中xml标签的匹配

　　现在有一个需求，比如给定如下数据：

0-0-0 0:0:0 #### the 68th annual golden globe awards ####  the king s speech earns 7 nominations  ####  <LOCATION>LOS ANGELES</LOCATION> <ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION> historical drama British king stammer beat competitors Tuesday grab seven nominations Golden Globe Awards nominations included best film drama nod contested award organizers said films competing best picture <ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION> earned nominations best performance actor olin <PERSON>Firth</PERSON> best performance actress <PERSON>Helena Bonham</PERSON> arter best supporting actor <PERSON>Geoffrey Rush</PERSON> best director <PERSON>Tom Hooper</PERSON> best screenplay <PERSON>David Seidler</PERSON> best movie score <ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION> earned nods apiece Black Swan Inception Kids Right tied place movie race nominations best motion picture comedy musical category <ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION> compete Nominated best actor motion picture olin <ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION> best actress motion picture nominees <PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON> Rabbit Hole <PERSON>Jennifer Lawrence</PERSON> <ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION> categories Glee nominee nods followed Rock Boardwalk Empire Dexter Good Wife Mad Men Modern Family Pillars Earth Temple <PERSON>Grandin</PERSON> tied nods apiece awards announced Jan 

 

　　要求按行把<></>标签内的字符串中的空格替换成下划线_，并且将数据转换形式，例：<X>A B C</X>需要转换成A_B_C/X

　　由于正则表达式匹配是贪婪模式，即尽可能匹配到靠后，那么就非常麻烦，而且仅仅是用?是无法真正保证是非贪婪的。所以需要在正则匹配时给之前匹配好的字符串标一个名字。

python下，正则最终写出来是这样：

LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')

　　接下来我们需要做是在原字符串中找出对应的子串，并且记下他们的位置，接下来就是预处理出需要替换成的样子，再用一个正则就好了。

LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

　　对字符串集合做整次的map，对每一个字符串进行匹配，再吧这两部分匹配结果zip在一起，就可以获得一个start-end的tuple，大致这样。

('<LOCATION>LOS ANGELES</LOCATION>', 'LOS_ANGELES/LOCATION')
('<ORGANIZATION>Dec Xinhua Kings Speech</ORGANIZATION>', 'Dec_Xinhua_Kings_Speech/ORGANIZATION')
('<ORGANIZATION>Social Network Black Swan Fighter Inception Kings Speech</ORGANIZATION>', 'Social_Network_Black_Swan_Fighter_Inception_Kings_Speech/ORGANIZATION')
('<PERSON>Firth</PERSON>', 'Firth/PERSON')
('<PERSON>Helena Bonham</PERSON>', 'Helena_Bonham/PERSON')
('<PERSON>Geoffrey Rush</PERSON>', 'Geoffrey_Rush/PERSON')
('<PERSON>Tom Hooper</PERSON>', 'Tom_Hooper/PERSON')
('<PERSON>David Seidler</PERSON>', 'David_Seidler/PERSON')
('<ORGANIZATION>Alexandre Desplat Social Network Fighter</ORGANIZATION>', 'Alexandre_Desplat_Social_Network_Fighter/ORGANIZATION')
('<ORGANIZATION>Alice Wonderland Burlesque Kids Right Red Tourist</ORGANIZATION>', 'Alice_Wonderland_Burlesque_Kids_Right_Red_Tourist/ORGANIZATION')
('<ORGANIZATION>Firth Kings Speech James Franco Hours Ryan Gosling Blue Valentine Mark Wahlberg Fighter Jesse Eisenberg Social Network</ORGANIZATION>', 'Firth_Kings_Speech_James_Franco_Hours_Ryan_Gosling_Blue_Valentine_Mark_Wahlberg_Fighter_Jesse_Eisenberg_Social_Network/ORGANIZATION')
('<PERSON>Halle Berry Frankie Alice Nicole Kidman</PERSON>', 'Halle_Berry_Frankie_Alice_Nicole_Kidman/PERSON')
('<PERSON>Jennifer Lawrence</PERSON>', 'Jennifer_Lawrence/PERSON')
('<ORGANIZATION>Winters Bone Natalie Portman Black Swan Michelle Williams Blue Valentine TV</ORGANIZATION>', 'Winters_Bone_Natalie_Portman_Black_Swan_Michelle_Williams_Blue_Valentine_TV/ORGANIZATION')
('<PERSON>Grandin</PERSON>', 'Grandin/PERSON')
('<LOCATION>BEIJING</LOCATION>', 'BEIJING/LOCATION')
('<ORGANIZATION>Xinhua Sanlu Group</ORGANIZATION>', 'Xinhua_Sanlu_Group/ORGANIZATION')
('<LOCATION>Gansu</LOCATION>', 'Gansu/LOCATION')
('<ORGANIZATION>Sanlu</ORGANIZATION>', 'Sanlu/ORGANIZATION')

　　处理的代码如下：

def read_file(path):
    if not os.path.exists(path):
        print 'path : \''+ path + '\' not find.'
        return []
    content = ''
    try:
        with open(path, 'r') as fp:
            content += reduce(lambda x,y:x+y, fp)
    finally:
        fp.close()
    return content.split('\n')

def get_label(each):
    pair = zip(LABEL_PATTERN.findall(each),
                         map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))
    return map(lambda x: (x[0][0], x[1]), pair)

src = read_file(FILE_PATH)
pattern = map(get_label, src)

 

　　接下来简单处理以下就好：

for i in range(0, len(src)):
    for pat in pattern[i]:
        src[i] = re.sub(pat[0], pat[1], src[i])

 

　　所有代码：

# -*- coding: utf-8 -*-
import re
import os

# FILE_PATH = '/home/kirai/workspace/sina_news_process/disworded_sina_news_attr_handled.txt'
FILE_PATH = '/home/kirai/workspace/sina_news_process/test.txt'
LABEL_PATTERN = re.compile('(<(?P<label>\S+)>.+?</(?P=label)>)')
LABEL_CONTENT_PATTERN = re.compile('<(?P<label>\S+)>(.*?)</(?P=label)>')

def read_file(path):
    if not os.path.exists(path):
        print 'path : \''+ path + '\' not find.'
        return []
    content = ''
    try:
        with open(path, 'r') as fp:
            content += reduce(lambda x,y:x+y, fp)
    finally:
        fp.close()
    return content.split('\n')

def get_label(each):
    pair = zip(LABEL_PATTERN.findall(each),
                         map(lambda x: x[1].replace(' ', '_')+'/'+x[0], LABEL_CONTENT_PATTERN.findall(each)))
    return map(lambda x: (x[0][0], x[1]), pair)

src = read_file(FILE_PATH)
pattern = map(get_label, src)

for i in range(0, len(src)):
    for pat in pattern[i]:
        src[i] = re.sub(pat[0], pat[1], src[i])