Python 3.4中的多处理与多线程与asyncio

时间:2021-07-29 18:13:23

I found that in Python 3.4 there are few different libraries for multiprocessing/threading: multiprocessing vs threading vs asyncio.

我发现在Python 3.4中,几乎没有用于多处理/线程的不同库:多处理与线程和asyncio。

But I don't know which one to use or is the "recommended one". Do they do the same thing, or are different? If so, which one is used for what? I want to write a program that uses multicores in my computer. But I don't know which library I should learn.

但我不知道使用哪一个或是“推荐的”。他们做同样的事情,还是不同?如果是这样,哪一个用于什么?我想编写一个在我的计算机中使用多核的程序。但我不知道应该学习哪个图书馆。

1 个解决方案

#1


31  

They are intended for (slightly) different purposes and/or requirements. CPython (a typical, mainline Python implementation) still has the global interpreter lock so a multi-threaded application (a standard way to implement parallel processing nowadays) is suboptimal. That's why multiprocessing may be preferred over threading. But not every problem may be effectively split into [almost independent] pieces, so there may be a need in heavy interprocess communications. That's why multiprocessing may not be preferred over threading in general.

它们旨在用于(略微)不同的目的和/或要求。 CPython(典型的主线Python实现)仍然具有全局解释器锁,因此多线程应用程序(现在实现并行处理的标准方法)不是最理想的。这就是多处理可能比线程更受欢迎的原因。但并非每个问题都可以有效地分解为[几乎独立的]部分,因此可能需要进行繁重的进程间通信。这就是为什么多处理一般不会优先于线程化的原因。

asyncio (this technique is available not only in Python, other languages and/or frameworks also have it, e.g. Boost.ASIO) is a method to effectively handle a lot of I/O operations from many simultaneous sources w/o need of parallel code execution. So it's just a solution (a good one indeed!) for a particular task, not for parallel processing in general.

asyncio(这种技术不仅可以在Python中使用,其他语言和/或框架也可以使用它,例如Boost.ASIO)是一种从需要并行代码的许多同时源有效处理大量I / O操作的方法执行。所以它只是一个特定任务的解决方案(确实是一个好的!),而不是一般的并行处理。

#1


31  

They are intended for (slightly) different purposes and/or requirements. CPython (a typical, mainline Python implementation) still has the global interpreter lock so a multi-threaded application (a standard way to implement parallel processing nowadays) is suboptimal. That's why multiprocessing may be preferred over threading. But not every problem may be effectively split into [almost independent] pieces, so there may be a need in heavy interprocess communications. That's why multiprocessing may not be preferred over threading in general.

它们旨在用于(略微)不同的目的和/或要求。 CPython(典型的主线Python实现)仍然具有全局解释器锁,因此多线程应用程序(现在实现并行处理的标准方法)不是最理想的。这就是多处理可能比线程更受欢迎的原因。但并非每个问题都可以有效地分解为[几乎独立的]部分,因此可能需要进行繁重的进程间通信。这就是为什么多处理一般不会优先于线程化的原因。

asyncio (this technique is available not only in Python, other languages and/or frameworks also have it, e.g. Boost.ASIO) is a method to effectively handle a lot of I/O operations from many simultaneous sources w/o need of parallel code execution. So it's just a solution (a good one indeed!) for a particular task, not for parallel processing in general.

asyncio(这种技术不仅可以在Python中使用,其他语言和/或框架也可以使用它,例如Boost.ASIO)是一种从需要并行代码的许多同时源有效处理大量I / O操作的方法执行。所以它只是一个特定任务的解决方案(确实是一个好的!),而不是一般的并行处理。