如何用AWS弹性豆茎来管理一个工人?

I am launching a django application on aws elastic beanstalk. I'd like to run background task or worker in order order to run celery.

我将在aws弹性豆茎上启动一个django应用程序。我想运行后台任务或工人，以便运行芹菜。

I can not find if it is possible or not. If yes how it could be achieve.

我找不到它是否可能。如果是，如何实现。

Here is what I am doing right now but this is producing every time an event type error.

这是我现在正在做的，但每次发生事件类型错误时，都会产生这种错误。

container_commands:
  01_syncdb:
    command: "django-admin.py syncdb --noinput"
    leader_only: true
  50_sqs_email:
    command: "./manage.py celery worker --loglevel=info"
    leader_only: true

Thanks for the help,

谢谢你的帮助,

2 个解决方案

#1

As @chris-wheadon suggested in his comment, you should try to run celery as a deamon in the background. AWS Elastic Beanstalk uses supervisord already to run some deamon processes. So you can leverage that to run celeryd and avoid creating a custom AMI for this. It works nicely for me.

正如@chris-wheadon在他的评论中所建议的那样，你应该试着在背景中使用芹菜作为deamon。AWS弹性豆茎采用已有的监管流程。因此，您可以利用它来运行celeryd，并避免为此创建自定义AMI。它对我很好。

What I do is to programatically add a celeryd config file to the instance after the app is deployed to it by EB. The tricky part is that the file needs to set the required environmental variables for the deamon (such as AWS access keys if you use S3 or other services in your app).

我所做的是在应用程序被EB部署到这个应用程序后，将一个celeryd配置文件添加到实例中。棘手的是，该文件需要为deamon设置所需的环境变量(比如如果在应用程序中使用S3或其他服务，则需要设置AWS访问密钥)。

Below there is a copy of the script that I use, add this script to your .ebextensions folder that configures your EB environment.

下面是我使用的脚本的一个副本，将这个脚本添加到您的. ebex紧张文件夹中，它可以配置您的EB环境。

The setup script creates a file in the undocumented /opt/elasticbeanstalk/hooks/appdeploy/post/ folder that lives on all EB instances. Any shell script in there will be executed post deployment. The shell script that is placed there works as follows:

安装脚本在所有EB实例上创建了一个文件，该文件存在于未归档的/opt/ ticbeanstalk/hook /appdeploy/post/文件夹中。其中的任何shell脚本都将在部署后执行。放置在那里的shell脚本如下所示:

In the celeryenv variable, the virutalenv environment is stored in a format that follows the supervisord notation. This is a comma separated list of env variables.
在celeryenv变量中，virutalenv环境以遵循主符号的格式存储。这是一个用逗号分隔的env变量列表。
Then the script creates a variable celeryconf that contains the configuration file as a string, which includes the previously parsed env variables.
然后脚本创建一个变量celeryconf，该变量将配置文件作为字符串包含在前面解析的env变量中。
This variable is then piped into a file called celeryd.conf, a supervisord configuration file for the celery daemon.
然后将该变量导入一个名为celeryd的文件。conf是芹菜守护进程的一个主配置文件。
Finally, the path to the newly created config file is added to the main supervisord.conf file, if it is not already there.
最后，将新创建的配置文件的路径添加到主监管文件中。conf文件，如果不存在的话。

Here is a copy of the script:

这是剧本的副本:

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash

      # Get django environment variables
      celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
      celeryenv=${celeryenv%?}

      # Create celery configuraiton script
      celeryconf="[program:celeryd]
      ; Set full path to celery program if using virtualenv
      command=/opt/python/run/venv/bin/celery worker -A myappname --loglevel=INFO

      directory=/opt/python/current/app
      user=nobody
      numprocs=1
      stdout_logfile=/var/log/celery-worker.log
      stderr_logfile=/var/log/celery-worker.log
      autostart=true
      autorestart=true
      startsecs=10

      ; Need to wait for currently executing tasks to finish at shutdown.
      ; Increase this if you have very long running tasks.
      stopwaitsecs = 600

      ; When resorting to send SIGKILL to the program to terminate it
      ; send SIGKILL to its whole process group instead,
      ; taking care of its children as well.
      killasgroup=true

      ; if rabbitmq is supervised, set its priority higher
      ; so it starts first
      priority=998

      environment=$celeryenv"

      # Create the celery supervisord conf script
      echo "$celeryconf" | tee /opt/python/etc/celery.conf

      # Add configuration script to supervisord conf (if not there already)
      if ! grep -Fxq "[include]" /opt/python/etc/supervisord.conf
          then
          echo "[include]" | tee -a /opt/python/etc/supervisord.conf
          echo "files: celery.conf" | tee -a /opt/python/etc/supervisord.conf
      fi

      # Reread the supervisord config
      supervisorctl -c /opt/python/etc/supervisord.conf reread

      # Update supervisord in cache without restarting all services
      supervisorctl -c /opt/python/etc/supervisord.conf update

      # Start/Restart celeryd through supervisord
      supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd

#2

I was trying to do something similar in PHP however for whatever reason I couldn't keep the worker running. I switched to a AMI on an EC2 server and have had success ever since.

我试图在PHP中做一些类似的事情，但是出于某种原因，我无法让工作人员继续运行。我在EC2服务器上切换到AMI，从那以后我就成功了。

#1