Python多处理apply_async永远不会在Windows 7上返回结果

I am trying to follow a very simple multiprocessing example:

我试图遵循一个非常简单的多处理示例：

import multiprocessing as mp

def cube(x):
    return x**3

pool    = mp.Pool(processes=2)
results = [pool.apply_async(cube, args=x) for x in range(1,7)]

However, on my windows machine, I am not able to get the result (on ubuntu 12.04LTS it runs perfectly).

但是，在我的Windows机器上，我无法获得结果（在ubuntu 12.04LTS上运行完美）。

If I inspect results, I see the following:

如果我检查结果，我会看到以下内容：

[<multiprocessing.pool.ApplyResult object at 0x01FF0910>,
 <multiprocessing.pool.ApplyResult object at 0x01FF0950>,
 <multiprocessing.pool.ApplyResult object at 0x01FF0990>,
 <multiprocessing.pool.ApplyResult object at 0x01FF09D0>,
 <multiprocessing.pool.ApplyResult object at 0x01FF0A10>,
 <multiprocessing.pool.ApplyResult object at 0x01FF0A50>]

If I run results[0].ready() I always get False.

如果我运行结果[0] .ready（）我总是得到False。

If I run results[0].get() the python interpreter freezes, waiting to get the result that never comes.

如果我运行结果[0] .get（），python解释器会冻结，等待获得永远不会出现的结果。

The example is as simple as it gets, so I am thinking this is a low level bug relating to the OS (I am on Windows 7). But perhaps someone else has a better idea?

这个例子很简单，所以我认为这是一个与操作系统相关的低级错误（我在Windows 7上）。但也许其他人有更好的主意？

2 个解决方案

#1

There are a couple of mistakes here. First, you must declare the Pool inside an if __name__ == "__main__": guard when running on Windows. Second, you have to pass the args keyword argument a sequence, even if you're only passing one argument. So putting that together:

这里有几个错误。首先，在Windows上运行时，必须在if __name__ ==“__ main__”：guard中声明池。其次，你必须传递一个序列的args关键字参数，即使你只传递一个参数。所以把它放在一起：

import multiprocessing as mp

def cube(x):
    return x**3

if __name__ == "__main__":
    pool    = mp.Pool(processes=2)
    results = [pool.apply_async(cube, args=(x,)) for x in range(1,7)]
    print([result.get() for result in results])

Output:

输出：

[1, 8, 27, 64, 125, 216]

Edit:

编辑：

Oh, as moarningsun mentions, multiprocessing does not work well in the interactive interpreter:

哦，正如moarningsun提到的，多处理在交互式解释器中不能很好地工作：

Note

注意

Functionality within this package requires that the __main__ module be importable by the children. This is covered in Programming guidelines however it is worth pointing out here. This means that some examples, such as the multiprocessing.Pool examples will not work in the interactive interpreter.

此程序包中的功能要求__main__模块可由子项导入。这在编程指南中有所涉及，但值得在此指出。这意味着某些示例（例如multiprocessing.Pool示例）在交互式解释器中不起作用。

So you'll need to actually execute the code as a script to test it properly.

因此，您需要将代码实际执行为脚本才能正确测试。

#2

I was running python 3 and the IDE was spyder in anaconda (windows ) and so this trick doesn't work for me. I tried a lot but couldn't make any difference. I got the reason for my problem and is the same listed by dano in his note. But after a long day of searching I got some solution and it helped me to run the same code my windows machine. This website helped me to get the solution:

我正在运行python 3并且IDE在anaconda（windows）中是spyder，因此这个技巧对我不起作用。我尝试了很多，但没有任何区别。我得到了我的问题的原因，并在他的笔记中由dano列出。但经过漫长的一天搜索，我得到了一些解决方案，它帮助我运行相同的代码我的Windows机器。这个网站帮助我获得了解决方案：

http://python.6.x6.nabble.com/Multiprocessing-Pool-woes-td5047050.html

http://python.6.x6.nabble.com/Multiprocessing-Pool-woes-td5047050.html

Since I was using the python 3, I changed the program a little like this:

由于我使用的是python 3，我改变了程序有点像这样：

from types import FunctionType
import marshal

def _applicable(*args, **kwargs):
  name = kwargs['__pw_name']
  code = marshal.loads(kwargs['__pw_code'])
  gbls = globals() #gbls = marshal.loads(kwargs['__pw_gbls'])
  defs = marshal.loads(kwargs['__pw_defs'])
  clsr = marshal.loads(kwargs['__pw_clsr'])
  fdct = marshal.loads(kwargs['__pw_fdct'])
  func = FunctionType(code, gbls, name, defs, clsr)
  func.fdct = fdct
  del kwargs['__pw_name']
  del kwargs['__pw_code']
  del kwargs['__pw_defs']
  del kwargs['__pw_clsr']
  del kwargs['__pw_fdct']
  return func(*args, **kwargs)

def make_applicable(f, *args, **kwargs):
  if not isinstance(f, FunctionType): raise ValueError('argument must be a function')
  kwargs['__pw_name'] = f.__name__  # edited
  kwargs['__pw_code'] = marshal.dumps(f.__code__)   # edited
  kwargs['__pw_defs'] = marshal.dumps(f.__defaults__)  # edited
  kwargs['__pw_clsr'] = marshal.dumps(f.__closure__)  # edited
  kwargs['__pw_fdct'] = marshal.dumps(f.__dict__)   # edited
  return _applicable, args, kwargs

def _mappable(x):
  x,name,code,defs,clsr,fdct = x
  code = marshal.loads(code)
  gbls = globals() #gbls = marshal.loads(gbls)
  defs = marshal.loads(defs)
  clsr = marshal.loads(clsr)
  fdct = marshal.loads(fdct)
  func = FunctionType(code, gbls, name, defs, clsr)
  func.fdct = fdct
  return func(x)

def make_mappable(f, iterable):
  if not isinstance(f, FunctionType): raise ValueError('argument must be a function')
  name = f.__name__    # edited
  code = marshal.dumps(f.__code__)   # edited
  defs = marshal.dumps(f.__defaults__)  # edited
  clsr = marshal.dumps(f.__closure__)  # edited
  fdct = marshal.dumps(f.__dict__)  # edited
  return _mappable, ((i,name,code,defs,clsr,fdct) for i in iterable)

After this function , the above problem code is also changed a little like this:

在这个函数之后，上面的问题代码也改变了一点：

from multiprocessing import Pool
from poolable import make_applicable, make_mappable

def cube(x):
  return x**3

if __name__ == "__main__":
  pool    = Pool(processes=2)
  results = [pool.apply_async(*make_applicable(cube,x)) for x in range(1,7)]
  print([result.get(timeout=10) for result in results])

And I got the output as :

我得到的输出为：

[1, 8, 27, 64, 125, 216]

I am thinking that this post may be useful for some of the windows users.

我认为这篇文章可能对一些Windows用户有用。