使用QueueHandler和QueueListener进行Python日志记录（logutils）

Logging is needed for a multiprocess python app. Using queues appears to be the best solution. The logutils library provides this.

多进程python应用程序需要记录。使用队列似乎是最好的解决方案。 logutils库提供了这个功能。

Is it possible to set the log levels for two handler independently? For example, in the test below I would like STREAM 1 to have WARNING messages and STREAM 2 to have INFO messages. In the test log at the end of the code an INFO message is created that should not output to the console from STREAM 1 handler (WARNING). However, it outputs to both handlers.

是否可以独立设置两个处理程序的日志级别?例如,在下面的测试中,我希望STREAM 1有WARNING消息而STREAM 2有INFO消息。在代码末尾的测试日志中,创建了一条INFO消息,该消息不应从STREAM 1处理程序(WARNING)输出到控制台。但是,它输出给两个处理程序。

For reference, I have been using this page http://plumberjack.blogspot.co.uk/2010/09/improved-queuehandler-queuelistener.html by Vinay Sajip who is the author of the library.

作为参考,我一直在使用这个页面的http://plumberjack.blogspot.co.uk/2010/09/improved-queuehandler-queuelistener.html作为图书馆的作者Vinay Sajip。

# System imports
import logging
import logging.handlers
try:
    import Queue as queue
except ImportError:
    import queue

# Custom imports
from logutils.queue import QueueHandler, QueueListener

# Get queue
q = queue.Queue(-1)

# Setup stream handler 1 to output WARNING to console
h1 = logging.StreamHandler()
f1 = logging.Formatter('STREAM 1 WARNING: %(threadName)s: %(message)s')
h1.setFormatter(f1)
h1.setLevel(logging.WARNING) # NOT WORKING. This should log >= WARNING

# Setup stream handler 2 to output INFO to console
h2 = logging.StreamHandler()
f2 = logging.Formatter('STREAM 2 INFO: %(threadName)s: %(message)s')
h2.setFormatter(f2)
h2.setLevel(logging.INFO) # NOT WORKING. This should log >= WARNING

# Start queue listener using the stream handler above
ql = QueueListener(q, h1, h2)
ql.start()

# Create log and set handler to queue handle
root = logging.getLogger()
root.setLevel(logging.DEBUG) # Log level = DEBUG
qh = QueueHandler(q)
root.addHandler(qh)

root.info('Look out!') # Create INFO message

ql.stop()

2 个解决方案

#1

This is a limitation in the implementation of the QueueListener.handle() method. This is currently:

这是QueueListener.handle()方法实现的限制。这是目前:

def handle(self, record):
    record = self.prepare(record)
    for handler in self.handlers:
        handler.handle(record)

To do what you want, it should be

要做你想做的事,它应该是

def handle(self, record):
    record = self.prepare(record)
    for handler in self.handlers:
        # CHANGED HERE TO ADD A CONDITION TO CHECK THE HANDLER LEVEL
        if record.levelno >= handler.level:
            handler.handle(record)

I will fix this at some point because I think this is better, but for now you can subclass QueueListener and override the handle method in the subclass.

我会在某些时候解决这个问题,因为我认为这样更好,但是现在你可以继承QueueListener并覆盖子类中的handle方法。

#2

I used this subclass to override the queue listener class. The two other methods addHandler and removeHandler allow for addition and removal of handlers. The CustomQueueListener should be used just like QueueListener. Follow the other logging examples for how to use addHandler() and removeHandler().

我使用此子类来覆盖队列侦听器类。另外两个方法addHandler和removeHandler允许添加和删除处理程序。 CustomQueueListener应该像QueueListener一样使用。请关注其他日志记录示例,了解如何使用addHandler()和removeHandler()。

class CustomQueueListener(QueueListener):
    def __init__(self, queue, *handlers):
        super(CustomQueueListener, self).__init__(queue, *handlers)
        """
        Initialise an instance with the specified queue and
        handlers.
        """
        # Changing this to a list from tuple in the parent class
        self.handlers = list(handlers)

    def handle(self, record):
        """
        Override handle a record.

        This just loops through the handlers offering them the record
        to handle.

        :param record: The record to handle.
        """
        record = self.prepare(record)
        for handler in self.handlers:
            if record.levelno >= handler.level: # This check is not in the parent class
                handler.handle(record)

    def addHandler(self, hdlr):
        """
        Add the specified handler to this logger.
        """
        if not (hdlr in self.handlers):
            self.handlers.append(hdlr)

    def removeHandler(self, hdlr):
        """
        Remove the specified handler from this logger.
        """
        if hdlr in self.handlers:
            hdlr.close()
            self.handlers.remove(hdlr)

#1