Even though Python and Ruby have one kernel thread per interpreter thread, they have a global interpreter lock (GIL) that is used to protect potentially shared data structures, so this inhibits multi-processor execution. Even though the portions in those languajes that are written in C or C++ can be free-threaded, that's not possible with pure interpreted code unless you use multiple processes. What's the best way to achieve this? Using FastCGI? Creating a cluster or a farm of virtualized servers? Using their Java equivalents, JRuby and Jython?
尽管Python和Ruby每个解释器线程都有一个内核线程,但它们有一个全局解释器锁(GIL),用于保护潜在的共享数据结构,因此这会禁止多处理器执行。尽管用C或C ++编写的那些语言中的部分可以是*线程的,但除非使用多个进程,否则使用纯解释代码是不可能的。实现这一目标的最佳方法是什么?使用FastCGI?创建群集或虚拟化服务器场?使用他们的Java等价物,JRuby和Jython?
5 个解决方案
#1
4
I'm not totally sure which problem you want so solve, but if you deploy your python/django application via an apache prefork MPM using mod_python apache will start several worker processes for handling different requests.
我不完全确定你想要解决哪个问题,但如果你通过apache prefork部署你的python / django应用程序MPM使用mod_python apache将启动几个工作进程来处理不同的请求。
If one request needs so much resources, that you want to use multiple cores have a look at pyprocessing. But I don't think that would be wise.
如果一个请求需要这么多资源,那么您想要使用多个核心就可以看一下pyprocessing。但我不认为这是明智的。
#2
4
The 'standard' way to do this with rails is to run a "pack" of Mongrel instances (ie: 4 copies of the rails application) and then use apache or nginx or some other piece of software to sit in front of them and act as a load balancer.
使用rails执行此操作的“标准”方法是运行Mongrel实例的“包”(即:rails应用程序的4个副本),然后使用apache或nginx或其他一些软件坐在它们前面并执行操作作为负载均衡器。
This is probably how it's done with other ruby frameworks such as merb etc, but I haven't used those personally.
这可能是它与其他ruby框架如merb等完成的方式,但我没有亲自使用过它们。
The OS will take care of running each mongrel on it's own CPU.
操作系统将负责在其自己的CPU上运行每个mongrel。
If you install mod_rails aka phusion passenger it will start and stop multiple copies of the rails process for you as well, so it will end up spreading the load across multiple CPUs/cores in a similar way.
如果您安装mod_rails aka phusion passenger,它也会为您启动和停止rails进程的多个副本,因此最终会以类似的方式将负载分散到多个CPU /核心。
#3
1
Use an interface that runs each response in a separate interpreter, such as mod_wsgi for Python. This lets multi-threading be used without encountering the GIL.
使用在单独的解释器中运行每个响应的接口,例如Python的mod_wsgi。这样可以在不遇到GIL的情况下使用多线程。
EDIT: Apparently, mod_wsgi no longer supports multiple interpreters per process because idiots couldn't figure out how to properly implement extension modules. It still supports running requests in separate processes FastCGI-style, though, so that's apparently the current accepted solution.
编辑:显然,mod_wsgi不再支持每个进程多个解释器,因为白痴无法弄清楚如何正确实现扩展模块。它仍然支持在单独的进程FastCGI样式中运行请求,因此这显然是当前接受的解决方案。
#4
1
In Python and Ruby it is only possible to use multiple cores, is to spawn new (heavyweight) processes. The Java counterparts inherit the possibilities of the Java platform. You could imply use Java threads. That is for example a reason why sometimes (often) Java Application Server like Glassfish are used for Ruby on Rails applications.
在Python和Ruby中,只能使用多个核心,就是产生新的(重量级)进程。 Java对应物继承了Java平台的可能性。你可以暗示使用Java线程。这就是为什么有时(通常)像Glassfish这样的Java Application Server被用于Ruby on Rails应用程序的原因。
#5
0
For Python, the PyProcessing project allows you to program with processes much like you would use threads. It is included in the standard library of the recently released 2.6 version as multiprocessing. The module has many features for establishing and controlling access to shared data structures (queues, pipes, etc.) and support for common idioms (i.e. managers and worker pools).
对于Python,PyProcessing项目允许您使用与使用线程类似的进程进行编程。它作为多处理包含在最近发布的2.6版本的标准库中。该模块具有许多功能,用于建立和控制对共享数据结构(队列,管道等)的访问以及对常见习语(即管理者和工作者池)的支持。
#1
4
I'm not totally sure which problem you want so solve, but if you deploy your python/django application via an apache prefork MPM using mod_python apache will start several worker processes for handling different requests.
我不完全确定你想要解决哪个问题,但如果你通过apache prefork部署你的python / django应用程序MPM使用mod_python apache将启动几个工作进程来处理不同的请求。
If one request needs so much resources, that you want to use multiple cores have a look at pyprocessing. But I don't think that would be wise.
如果一个请求需要这么多资源,那么您想要使用多个核心就可以看一下pyprocessing。但我不认为这是明智的。
#2
4
The 'standard' way to do this with rails is to run a "pack" of Mongrel instances (ie: 4 copies of the rails application) and then use apache or nginx or some other piece of software to sit in front of them and act as a load balancer.
使用rails执行此操作的“标准”方法是运行Mongrel实例的“包”(即:rails应用程序的4个副本),然后使用apache或nginx或其他一些软件坐在它们前面并执行操作作为负载均衡器。
This is probably how it's done with other ruby frameworks such as merb etc, but I haven't used those personally.
这可能是它与其他ruby框架如merb等完成的方式,但我没有亲自使用过它们。
The OS will take care of running each mongrel on it's own CPU.
操作系统将负责在其自己的CPU上运行每个mongrel。
If you install mod_rails aka phusion passenger it will start and stop multiple copies of the rails process for you as well, so it will end up spreading the load across multiple CPUs/cores in a similar way.
如果您安装mod_rails aka phusion passenger,它也会为您启动和停止rails进程的多个副本,因此最终会以类似的方式将负载分散到多个CPU /核心。
#3
1
Use an interface that runs each response in a separate interpreter, such as mod_wsgi for Python. This lets multi-threading be used without encountering the GIL.
使用在单独的解释器中运行每个响应的接口,例如Python的mod_wsgi。这样可以在不遇到GIL的情况下使用多线程。
EDIT: Apparently, mod_wsgi no longer supports multiple interpreters per process because idiots couldn't figure out how to properly implement extension modules. It still supports running requests in separate processes FastCGI-style, though, so that's apparently the current accepted solution.
编辑:显然,mod_wsgi不再支持每个进程多个解释器,因为白痴无法弄清楚如何正确实现扩展模块。它仍然支持在单独的进程FastCGI样式中运行请求,因此这显然是当前接受的解决方案。
#4
1
In Python and Ruby it is only possible to use multiple cores, is to spawn new (heavyweight) processes. The Java counterparts inherit the possibilities of the Java platform. You could imply use Java threads. That is for example a reason why sometimes (often) Java Application Server like Glassfish are used for Ruby on Rails applications.
在Python和Ruby中,只能使用多个核心,就是产生新的(重量级)进程。 Java对应物继承了Java平台的可能性。你可以暗示使用Java线程。这就是为什么有时(通常)像Glassfish这样的Java Application Server被用于Ruby on Rails应用程序的原因。
#5
0
For Python, the PyProcessing project allows you to program with processes much like you would use threads. It is included in the standard library of the recently released 2.6 version as multiprocessing. The module has many features for establishing and controlling access to shared data structures (queues, pipes, etc.) and support for common idioms (i.e. managers and worker pools).
对于Python,PyProcessing项目允许您使用与使用线程类似的进程进行编程。它作为多处理包含在最近发布的2.6版本的标准库中。该模块具有许多功能,用于建立和控制对共享数据结构(队列,管道等)的访问以及对常见习语(即管理者和工作者池)的支持。