Scenario
场景
I have a server process implemented in Python and gevent, which is connecting to a backend service over TCP/IP and serving many Web clients (~1000 per process) with fast changing dynamic content based on the changes delivered by the backend service. The service is running on Ubuntu 10.04 and won't run on Windows, so the target platform is fixed. We're using Python 2.6.6, currently.
我有一个在Python和gevent中实现的服务器进程，它通过TCP/IP连接到后端服务，并根据后端服务提供的更改，使用快速变化的动态内容为许多Web客户机(每个进程约1000个)提供服务。该服务在Ubuntu 10.04上运行，不会在Windows上运行，因此目标平台是固定的。目前我们使用的是Python 2.6.6。
Problem
问题
Serving the changes to that many clients can cause a lag in processing the changes sent by the backend, so my plan is to split the server into multiple processes. A number of worker processes would serve the Web clients, while the main process would still be connected to the backend service. I'm already using a separate pool of greenlets to serve the Web clients, but they need to be put into worker processes.
为许多客户端提供更改可能会导致在后端发送的更改处理滞后，因此我的计划是将服务器分割为多个进程。许多工作进程将服务于Web客户端，而主进程仍将连接到后端服务。我已经在使用一个单独的greenlet池来服务Web客户端，但是它们需要放到worker进程中。
Question
问题
Could you please point me to a working process pool implementation for gevent or figure out how can I use Python's own multiprocessing module with gevent the right way?
您能给我指出一个gevent的工作进程池实现吗?或者想想如何正确地使用Python自己的多处理模块和gevent ?
Restrictions
限制
I would like to avoid introducing Python threads into our processes, since that would give room for GIL contention, which would reduce performance by introducing latencies. So it would be a clean multiprocessing + gevent solution, if possible.
我希望避免在我们的进程中引入Python线程，因为这会给GIL争用留出空间，这会通过引入延迟降低性能。如果可能的话，这将是一个干净的多处理+ gevent解决方案。
2 个解决方案
#1
1
I recommend taking a look at Celery - distributed task processing system written in Python.
我建议看一下用Python编写的芹菜分布式任务处理系统。
The basic idea with celery is that you have a queue (either RabbitMQ or Redis), with workers written as Python processes, and exposed via the celeryd daemon. According to the celery documentation, celeryd supports running in gevent mode too (for network I/O bound processes), so presumably your worker code wouldn't need much modification to run in this environment.
芹菜的基本思想是，您有一个队列(RabbitMQ或Redis)，它以Python进程的形式编写工作者，并通过celeryd守护进程公开。根据芹菜文档，celeryd也支持在gevent模式下运行(对于网络I/O绑定的进程)，因此您的工作代码可能不需要太多修改就可以在这个环境中运行。
#2
2
I'd consider custom socket IPC or using ZeroMQ.
我将考虑自定义套接字IPC或使用ZeroMQ。
#1
1
I recommend taking a look at Celery - distributed task processing system written in Python.
我建议看一下用Python编写的芹菜分布式任务处理系统。
The basic idea with celery is that you have a queue (either RabbitMQ or Redis), with workers written as Python processes, and exposed via the celeryd daemon. According to the celery documentation, celeryd supports running in gevent mode too (for network I/O bound processes), so presumably your worker code wouldn't need much modification to run in this environment.
芹菜的基本思想是，您有一个队列(RabbitMQ或Redis)，它以Python进程的形式编写工作者，并通过celeryd守护进程公开。根据芹菜文档，celeryd也支持在gevent模式下运行(对于网络I/O绑定的进程)，因此您的工作代码可能不需要太多修改就可以在这个环境中运行。
#2
2
I'd consider custom socket IPC or using ZeroMQ.
我将考虑自定义套接字IPC或使用ZeroMQ。