如何在Python中使用多处理队列?

I'm having much trouble trying to understand just how the multiprocessing queue works on python and how to implement it. Lets say I have two python modules that access data from a shared file, let's call these two modules a writer and a reader. My plan is to have both the reader and writer put requests into two separate multiprocessing queues, and then have a third process pop these requests in a loop and execute as such.

我很难理解多处理队列如何处理python以及如何实现它。假设我有两个从共享文件访问数据的python模块，我们将这两个模块称为writer和reader。我的计划是让读写器将请求放入两个单独的多处理队列中，然后让第三个进程将这些请求放入一个循环中并按此方式执行。

My main problem is that I really don't know how to implement multiprocessing.queue correctly, you cannot really instantiate the object for each process since they will be separate queues, how do you make sure that all processes relate to a shared queue (or in this case, queues)

我的主要问题是我真的不知道如何实现多处理。如果队列是正确的，则不能实例化每个进程的对象，因为它们将是独立的队列，那么如何确保所有进程都与共享队列相关(在本例中，是队列)

2 个解决方案

#1

My main problem is that I really don't know how to implement multiprocessing.queue correctly, you cannot really instantiate the object for each process since they will be separate queues, how do you make sure that all processes relate to a shared queue (or in this case, queues)

我的主要问题是我真的不知道如何实现多处理。如果队列是正确的，则不能实例化每个进程的对象，因为它们将是独立的队列，那么如何确保所有进程都与共享队列相关(在本例中，是队列)

This is a simple example of a reader and writer sharing a single queue... The writer sends a bunch of integers to the reader; when the writer runs out of numbers, it sends 'DONE', which lets the reader know to break out of the read loop.

这是一个读者和作者共享一个队列的简单例子……作者向读者发送一串整数;当作者用完了数字，它会发送“完成”，让读者知道如何跳出读循环。

from multiprocessing import Process
from queue import Queue
import time

def reader(queue):
    ## Read from the queue
    while True:
        msg = queue.get()         # Read from the queue and do nothing
        if (msg == 'DONE'):
            break

def writer(count, queue):
    ## Write to the queue
    for ii in xrange(0, count):
        queue.put(ii)             # Write 'count' numbers into the queue
    queue.put('DONE')

if __name__=='__main__':
    for count in [10**4, 10**5, 10**6]:
        queue = Queue()   # reader() reads from queue
                          # writer() writes to queue
        reader_p = Process(target=reader, args=((queue),))
        reader_p.daemon = True
        reader_p.start()        # Launch reader() as a separate python process

        _start = time.time()
        writer(count, queue)    # Send a lot of stuff to reader()
        reader_p.join()         # Wait for the reader to finish
        print "Sending %s numbers to Queue() took %s seconds" % (count, 
            (time.time() - _start))

#2

in "from queue import Queue" there is no module called queue, instead multiprocessing should be used. Therefore, it should look like "from multiprocessing import Queue"

在“从队列导入队列”中，不存在称为队列的模块，而应该使用多处理。因此，它应该看起来像“来自多处理导入队列”

#1