Python的BaseHTTPServer，我如何捕获/捕获“坏掉的管道”错误?

I build a short url translator engine in Python, and I'm seeing a TON of "broken pipe" errors, and I'm curious how to trap it best when using the BaseHTTPServer classes. This isn't the entire code, but gives you an idea of what I'm doing so far:

我在Python中构建了一个简短的url翻译引擎，我看到了大量的“破损管道”错误，我很好奇如何在使用BaseHTTPServer类时更好地捕获它。这不是全部的代码，但是让你知道我目前在做什么:

    from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
    import memcache

    class clientThread(BaseHTTPRequestHandler):

            def do_GET(self):
                    content = None
                    http_code,response_txt,long_url = \
                            self.ag_trans_url(self.path,content,'GET')
                    self.http_output( http_code, response_txt, long_url )
                    return

            def http_output(self,http_code,response_txt,long_url):
                    self.send_response(http_code)
                    self.send_header('Content-type','text/plain')
                    if long_url:
                            self.send_header('Location', long_url)
                    self.end_headers()
                    if response_txt:
                            self.wfile.write(response_txt)
                    return

        def ag_trans_url(self, orig_short_url, post_action, getpost):
                short_url = 'http://foo.co' + orig_short_url

                # fetch it from memcache
                long_url = mc.get(short_url)

                # other magic happens to look it up from db if there was nothing
                # in memcache, etc
                return (302, None, log_url)

def populate_memcache()
        # connect to db, do lots of mc.set() calls

def main():
        populate_memcache()
        try:
                port = 8001
                if len(sys.argv) > 1:
                        port = int(sys.argv[1])
                server = HTTPServer(('',port), clientThread)
                #server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
                print '[',str(datetime.datetime.now()),'] short url processing has begun'

                server.serve_forever()
        except KeyboardInterrupt,SystemExit:
                print '^C received, shutting down server'
                server.socket.close()

The code itself works great, but started throwing errors almost immediately when in production:

代码本身工作很好，但是在生产过程中几乎立即开始抛出错误:

Traceback (most recent call last):
  File "/usr/lib/python2.5/SocketServer.py", line 222, in handle_request
    self.process_request(request, client_address)
  File "/usr/lib/python2.5/SocketServer.py", line 241, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.5/SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.5/SocketServer.py", line 522, in __init__
    self.handle()
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 310, in handle_one_request
    method()
  File "/opt/short_url_redirector/shorturl.py", line 38, in do_GET
    self.http_output( http_code, response_txt, long_url )
  File "/opt/short_url_redirector/shorturl.py", line 52, in http_output
    self.send_response(http_code)
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 370, in send_response
    self.send_header('Server', self.version_string())
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 376, in send_header
    self.wfile.write("%s: %s\r\n" % (keyword, value))
  File "/usr/lib/python2.5/socket.py", line 274, in write
    self.flush()
  File "/usr/lib/python2.5/socket.py", line 261, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')

The bulk of these errors seem to stem from having a problem calling the send_header() method where all I'm writing out is this:

这些错误的大部分似乎源于调用send_header()方法的问题，我在这里写的是:

self.send_header('Location', long_url)

So I'm curious where in my code to try to trap for this IO exception... do I write try/except calls around each of the self.send_header/self.end_headers/self.wfile.write calls? The other error I see from time to time is this one, but not sure which exception to watch to even catch this:

所以我很好奇，在我的代码中，试图为这个IO异常设置陷阱…do I write try/除了调用每个self。send_header/self。end_header/self。wfile。写电话吗?我不时看到的另一个错误是这个，但不确定要注意哪个例外:

Traceback (most recent call last):
  File "/usr/lib/python2.5/SocketServer.py", line 222, in handle_request
    self.process_request(request, client_address)
  File "/usr/lib/python2.5/SocketServer.py", line 241, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.5/SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.5/SocketServer.py", line 522, in __init__
    self.handle()
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "/usr/lib/python2.5/BaseHTTPServer.py", line 299, in handle_one_request
    self.raw_requestline = self.rfile.readline()
  File "/usr/lib/python2.5/socket.py", line 381, in readline
    data = self._sock.recv(self._rbufsize)
error: (104, 'Connection reset by peer')

3 个解决方案

#1

The "broken pipe" exception means that your code tried to write to a socket/pipe which the other end has closed. If the other end is a web browser, the user could have stopped the request. You can ignore the traceback; it does not indicate a serious problem. If you want to suppress the message, you can put a try ... except block around all of the code in your http_output function, and log the exception if you like.

“破裂管道”异常意味着您的代码试图写入到另一端已关闭的套接字/管道。如果另一端是web浏览器，则用户可以停止请求。你可以忽略回溯;这并不意味着一个严重的问题。如果你想要压制信息，你可以试试。除了您的http_output函数中的所有代码外，如果您愿意，也可以记录这个异常。

Additionally, if you want your HTTP server to process more than one request at a time, you need your server class to use one of the SocketServer.ForkingMixIn and SocketServer.ThreadingMixIn classes. Check the documentation of the SocketServer module for details.

另外，如果您希望您的HTTP服务器一次处理多个请求，那么您需要您的服务器类使用一个SocketServer。ForkingMixIn SocketServer。ThreadingMixIn类。详细检查SocketServer模块的文档。

Add: The "connection reset by peer" exception means that your code tried to read from a dead socket. If you want to suppress the traceback, you will need to extend the BaseHTTPServer class and override the handle_one_request method to add a try ... except block. You will need a new server class anyway, to implement the earlier suggestion about processing more than one request at a time.

添加:“对等重置”异常意味着您的代码试图从一个已死的套接字读取。如果您想要抑制traceback，您需要扩展BaseHTTPServer类并覆盖handle_one_request方法来添加try…除了块。无论如何，您将需要一个新的服务器类，以实现早期处理多个请求的建议。

#2

This appears to be a bug in SocketServer, see this link Python Bug: 14574

这在SocketServer中似乎是一个错误，请参见此链接Python bug: 14574。

A fix (works for me in Python 2.7) is to override the SocketServer.StreamRequestHandler finish() method, something like this:

修复(在Python 2.7中为我工作)是覆盖SocketServer。StreamRequestHandler finish()方法，如下所示:

...
def finish(self,*args,**kw):
  try:
    if not self.wfile.closed:
      self.wfile.flush()
      self.wfile.close()
  except socket.error:
    pass
  self.rfile.close()

  #Don't call the base class finish() method as it does the above
  #return SocketServer.StreamRequestHandler.finish(self)

#3

In my application, the error didn't occur in finish(), it occurred in handle(). This fix catches the broken pipe errors:

在我的应用程序中，错误没有发生在finish()中，它发生在handle()中。这一修复捕获了破裂的管道错误:

class MyHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

    ...

    def handle(self):
        try:
            BaseHTTPServer.BaseHTTPRequestHandler.handle(self)
        except socket.error:
            pass

#1

#2