使用线程和队列时如何处理异常？

If I have a program that uses threading and Queue, how do I get exceptions to stop execution? Here is an example program, which is not possible to stop with ctrl-c (basically ripped from the python docs).

如果我有一个使用线程和队列的程序,我如何获得停止执行的异常?这是一个示例程序,不能用ctrl-c停止(基本上是从python文档中删除)。

from threading import Thread
from Queue import Queue
from time import sleep

def do_work(item):
    sleep(0.5)
    print "working" , item

def worker():
        while True:
            item = q.get()
            do_work(item)
            q.task_done()

q = Queue()

num_worker_threads = 10

for i in range(num_worker_threads):
     t = Thread(target=worker)
    # t.setDaemon(True)
     t.start()

for item in range(1, 10000):
    q.put(item)

q.join()       # block until all tasks are done

1 个解决方案

#1

The simplest way is to start all the worker threads as daemon threads, then just have your main loop be

最简单的方法是将所有工作线程作为守护程序线程启动,然后将主循环设置为

while True:
    sleep(1)

Hitting Ctrl+C will throw an exception in your main thread, and all of the daemon threads will exit when the interpreter exits. This assumes you don't want to perform cleanup in all of those threads before they exit.

按Ctrl + C将在主线程中抛出异常,并且当解释器退出时,所有守护程序线程都将退出。这假设您不希望在退出之前在所有这些线程中执行清理。

A more complex way is to have a global stopped Event:

更复杂的方法是使全局停止事件:

stopped = Event()
def worker():
    while not stopped.is_set():
        try:
            item = q.get_nowait()
            do_work(item)
        except Empty:      # import the Empty exception from the Queue module
            stopped.wait(1)

Then your main loop can set the stopped Event to False when it gets a KeyboardInterrupt

然后你的主循环可以在它获得KeyboardInterrupt时将停止的Event设置为False

try:
    while not stopped.is_set():
        stopped.wait(1)
except KeyboardInterrupt:
    stopped.set()

This lets your worker threads finish what they're doing you want instead of just having every worker thread be a daemon and exit in the middle of execution. You can also do whatever cleanup you want.

这使您的工作线程可以完成他们想要的操作而不是让每个工作线程都成为守护进程并在执行过程中退出。你也可以做任何你需要的清理工作。

Note that this example doesn't make use of q.join() - this makes things more complex, though you can still use it. If you do then your best bet is to use signal handlers instead of exceptions to detect KeyboardInterrupts. For example:

请注意,此示例不使用q.join() - 这会使事情变得更复杂,尽管您仍然可以使用它。如果你这样做,那么最好的办法是使用信号处理程序而不是异常来检测KeyboardInterrupts。例如:

from signal import signal, SIGINT
def stop(signum, frame):
    stopped.set()
signal(SIGINT, stop)

This lets you define what happens when you hit Ctrl+C without affecting whatever your main loop is in the middle of. So you can keep doing q.join() without worrying about being interrupted by a Ctrl+C. Of course, with my above examples, you don't need to be joining, but you might have some other reason for doing so.

这使您可以定义在按Ctrl + C时发生的情况,而不会影响主循环中间的任何内容。所以你可以继续做q.join()而不用担心被Ctrl + C打断。当然,通过上面的示例,您不需要加入,但您可能还有其他原因要这样做。

#1