I have a strange issue with Celery.
我和Celery有一个奇怪的问题。
I am using RabbitMQ as message broker and result backend.
我使用RabbitMQ作为消息代理和结果后端。
Tasks are serialized via pickle, but they only get an id for a file in a database. They fetch it, do some work on it and write the result back to the database. I'm just storing the id in the result backend.
任务通过pickle序列化，但它们只获取数据库中文件的id。他们获取它，对它做一些工作并将结果写回数据库。我只是将id存储在结果后端。
I use a group to supply the tasks and don't run any subtasks from within it.
我使用一个组来提供任务，并且不从其中运行任何子任务。
I have one worker with concurrency=8 (prefork)
我有一个并发= 8的工人（prefork）
If I start the task, all 8 processes are working (100% cpu usage).
如果我启动任务，则所有8个进程都在工作（100％cpu使用率）。
After the first task finishes, the strange behavior begins. The process does not begin a new task. The task get's initialized (I used CELERYD_MAX_TASKS_PER_CHILD=1) but the run method doesn't get called.
第一个任务完成后，奇怪的行为开始了。该过程不会开始新任务。任务get初始化（我使用了CELERYD_MAX_TASKS_PER_CHILD = 1）但是没有调用run方法。
So the problem is, that not all processes are working all the time.
所以问题是，并非所有流程都在运行。
Tried many configuration settings but nothing changed this behavior.
尝试了许多配置设置，但没有改变此行为。
Do you have any idea?
你有什么主意吗？
It's not the database etc. Running message broker and database locally. Also had a look on the workers with flower, it says that most of the time round about 4 processes are active. Other tasks are reserved, but don't start.
它不是数据库等。本地运行消息代理和数据库。还看了一下有花的工人，它说大部分时间大约有4个过程都是活跃的。其他任务保留，但不启动。
Hope u can help me!
希望你能帮助我！
1 个解决方案
#1
12
Finally figured it out:
终于想通了：
It's just an option I had to put then starting the worker.
这只是我必须选择然后启动工人的一个选项。
Starting the worker with the -Ofair option did it!
使用-Ofair选项启动worker就可以了！
See: http://docs.celeryproject.org/en/latest/userguide/optimizing.html#prefork-pool-prefetch-settings
请参阅：http：//docs.celeryproject.org/en/latest/userguide/optimizing.html#prefork-pool-prefetch-settings
Thanks for your help :)
谢谢你的帮助 ：）
#1
12
Finally figured it out:
终于想通了：
It's just an option I had to put then starting the worker.
这只是我必须选择然后启动工人的一个选项。
Starting the worker with the -Ofair option did it!
使用-Ofair选项启动worker就可以了！
See: http://docs.celeryproject.org/en/latest/userguide/optimizing.html#prefork-pool-prefetch-settings
请参阅：http：//docs.celeryproject.org/en/latest/userguide/optimizing.html#prefork-pool-prefetch-settings
Thanks for your help :)
谢谢你的帮助 ：）