Amazon Web Service has now a worker tiers in their Elastic Beanstalk. But, it nevertheless confuse us who come from the days of Worker dyno.
Amazon Web Service现在在Elastic Beanstalk中有一个工作层。但是，它让我们混淆了那些来自工人dyno时代的人。
As a comparison, in Heroku, one can configure two dynos (something like processor?) each for web and worker. The web will work for any request, and will timeout normally at 15 secs. Thus, if you have a request that last more than that, your request will simply timed-out although not terminated per se. In that case, you should use worker and your web dyno should visit the endpoint several times per minutes (maybe) to check if there is any result to be brought back to the user. To make either worker or web dyno, what you need is just slide the slider and you are good to go. Sometimes, you may need a Procfile. But there is nothing fancy, or something really difficult, or confusing about it.
作为比较，在Heroku中，可以为web和worker配置两个dynos（类似处理器？）。网络将适用于任何请求，并将在15秒后正常超时。因此，如果您的请求持续时间超过此要求，您的请求将会暂时超时，但本身并未终止。在这种情况下，您应该使用worker并且您的web dyno应该每分钟（可能）多次访问端点以检查是否有任何结果被带回用户。要制作工作者或网络动态，你需要的只是滑动滑块，你就可以了。有时，您可能需要一个Procfile。但是没有什么花哨的东西，或者真的很难的东西，或者令人困惑的东西。
In AWS EBS (Elastic Beanstalk), since day 1 you hit eb init, you will be asked whether it is a Standard or Worker. When you hit Standard, it seems there is no way to make it as worker as well.
在AWS EBS（Elastic Beanstalk）中，从第1天开始您点击eb init，将询问您是标准版还是工作版。当你点击标准时，似乎没有办法让它成为工人。
In our situation, the worker and standard web is located under one application. So, how could we use an EBS instance both for worker and standard. Our worker is using sidekiq, and redis. Please, point to any guidance or help us in this matter.
在我们的情况下，工作者和标准Web位于一个应用程序下。那么，我们如何为工人和标准使用EBS实例。我们的工作人员正在使用sidekiq和redis。请指出任何指导或帮助我们解决此问题。
1 个解决方案
#1
22
AWS Elastic Beanstalk has two types of Environments - Web tier and Worker tier.
AWS Elastic Beanstalk有两种类型的环境 - Web层和工作层。
Web tier environments are meant for web applications - http/https request processing. You get one or more EC2 instances behind a load balancer. You can get other resources like database per your requirement. You can choose the platform you wish e.g. Ruby, Python, Java, Node.js, PHP, Docker.
Web层环境适用于Web应用程序 - http / https请求处理。您在负载均衡器后面获得一个或多个EC2实例。您可以根据需要获取数据库等其他资源。您可以选择您想要的平台，例如Ruby，Python，Java，Node.js，PHP，Docker。
Worker environments are meant for asynchronous message processing. When you create a worker environment you do not have a load balancer. All your EC2 instances are in an autoscaling group. All these instances are running a daemon which is polling a single SQS queue for messages. When a message is pulled by the daemon from the SQS queue, the daemon sends a HTTP Post request on localhost:80. You can configure the port but the important thing is that the daemon posts the message as an HTTP request on localhost. Your worker application is actually a web application that receives the post request and processes the message. After the message is successfully processed the worker daemon expects that your web application running on localhost returns a HTTP 200 OK response. The daemon then deletes the message from SQS queue. You can write your worker application for any platform just like standard web server applications - Ruby, Python, Java, Node.js, PHP, Docker.
工作线程环境用于异步消息处理。创建工作线程环境时，您没有负载均衡器。您的所有EC2实例都在自动缩放组中。所有这些实例都在运行守护程序，该守护程序轮询单个SQS队列以查找消息。当守护程序从SQS队列中提取消息时，守护程序会在localhost：80上发送HTTP Post请求。您可以配置端口，但重要的是守护程序将消息作为localhost上的HTTP请求发布。您的工作者应用程序实际上是一个Web应用程序，它接收发布请求并处理消息。成功处理消息后，worker守护程序期望在localhost上运行的Web应用程序返回HTTP 200 OK响应。守护进程然后从SQS队列中删除该消息。您可以为任何平台编写工作者应用程序，就像标准Web服务器应用程序一样 - Ruby，Python，Java，Node.js，PHP，Docker。
Based on my understanding of your usecase I would recommend creating two Elastic Beanstalk environments - one Standard and one Worker environment. The Standard web server receives HTTP requests and processes them synchronously. This environment puts the relevant data in an SQS queue. The second environment is a worker and the daemon running on this environment polls this SQS queue for messages. Your second environment is a web application that is NOT open to the internet. The worker daemon posts the messages as HTTP requests to your worker environment. Thus you can process long running workloads asynchronously using this second worker environment.
根据我对您的用例的理解，我建议创建两个Elastic Beanstalk环境 - 一个Standard环境和一个Worker环境。标准Web服务器接收HTTP请求并同步处理它们。此环境将相关数据放入SQS队列中。第二个环境是一个worker，在此环境中运行的守护程序会轮询此SQS队列以获取消息。您的第二个环境是一个不向互联网开放的Web应用程序。 worker守护程序将消息作为HTTP请求发布到您的工作环境。因此，您可以使用此第二个工作线程环境异步处理长时间运行的工作负载。
With worker environments you can use your own queues or Elastic Beanstalk can generate a queue for you. You can configure parameters like message visibility timeout, http connections based on your requirements or you can use the defaults.
使用工作线程环境，您可以使用自己的队列，或者Elastic Beanstalk可以为您生成队列。您可以根据需要配置消息可见性超时，http连接等参数，也可以使用默认值。
Below are some links that may be useful for you:
以下是一些可能对您有用的链接：
http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/
http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/
http://blogs.aws.amazon.com/application-management/post/Tx1Y8QSQRL1KQZC/Elastic-Beanstalk-Video-Tutorial-Worker-Tier
https://*.com/a/23942498/161628
https://*.com/a/23942498/161628
Does this meet your requirements? Please let me know if you have further questions.
这符合您的要求吗？如果您有其他问题，请告诉我。
Update
更新
You need to upload your source code at two places - once for the worker environment and once for the web server environment. If someone was starting from scratch then they might have two separate code bases. But I think in your case I think it should be perfectly fine to have a single code base shared between the two environments. Suppose your web request arrives at '/register', then the register() method in your application can post messages to an SQS queue and be done with the HTTP request. Now your worker environment will poll the SQS queue and post messages over HTTP on localhost to the URL '/async_register' which will invoke a method async_register() in your application and do the asynchronous processing. These two methods can live in the same source code bundle which can be shared by both the worker and web server environment. The code path taken by worker and web server will be different so that web server environments will invoke register() and worker environments will invoke the async_register() method.
您需要在两个位置上传源代码 - 一次用于工作环境，一次用于Web服务器环境。如果有人从头开始，那么他们可能有两个独立的代码库。但我认为在你的情况下，我认为在两个环境之间共享一个代码库应该是完全没问题的。假设您的Web请求到达'/ register'，那么应用程序中的register（）方法可以将消息发布到SQS队列并完成HTTP请求。现在，您的工作线程环境将轮询SQS队列，并通过localhost上的HTTP发送消息到URL'/ async_register'，它将在您的应用程序中调用方法async_register（）并执行异步处理。这两种方法可以存在于同一个源代码包中，可以由工作者和Web服务器环境共享。工作者和Web服务器采用的代码路径将不同，因此Web服务器环境将调用register（），而工作线程环境将调用async_register（）方法。
Another caveat is that HTTP requests sent by the worker daemon on localhost will contain an HTTP header - "User-Agent": "aws-sqsd/1.1". Read more here. So in your web application you can have a single listener to post requests on "/register" and depending on whether this header is present or not you invoke register() or async_register() methods internally.
另一个警告是，工作守护程序在localhost上发送的HTTP请求将包含HTTP标头 - “User-Agent”：“aws-sqsd / 1.1”。在这里阅读更多。因此，在您的Web应用程序中，您可以让一个侦听器在“/ register”上发布请求，并根据是否存在此标头，在内部调用register（）或async_register（）方法。
Also I think if you want to share the code base between the two environments you can upload the code base at only one place. Your environments are logically grouped into applications. So you can have a single application. You upload your source code to this application using the "CreateApplicationVersion" API call. Suppose you upload an application version with label 'v1'. You can now create a worker environment and a web server environment under the same application. When you create an environment you need to provide a version to deploy to your enviroment. In this case you can deploy v1 to both environments. So you will be sharing the same source code for both environments. When you have a new version "v2". You upload this version and then perform an update on both environments changing their version to "v2".
另外我想如果你想在两个环境之间共享代码库，你可以只在一个地方上传代码库。您的环境在逻辑上分组到应用程序中。所以你可以拥有一个应用程序。您使用“CreateApplicationVersion”API调用将源代码上载到此应用程序。假设您上传了标签为“v1”的应用程序版本。您现在可以在同一应用程序下创建工作线程环境和Web服务器环境。创建环境时，您需要提供一个版本以部署到您的环境中。在这种情况下，您可以将v1部署到两个环境。因此，您将为两种环境共享相同的源代码。当你有一个新版本“v2”。您上传此版本，然后在两个环境中将其版本更改为“v2”时执行更新。
The same version of the source code can be deployed to both environments. They will be running on different EC2 instances because one environment is dedicated for responding to web requests and one environment is dedicated for responding to asynchronous web requests (worker).
可以将相同版本的源代码部署到两个环境中。它们将在不同的EC2实例上运行，因为一个环境专用于响应Web请求，一个环境专用于响应异步Web请求（工作者）。
#1
22
AWS Elastic Beanstalk has two types of Environments - Web tier and Worker tier.
AWS Elastic Beanstalk有两种类型的环境 - Web层和工作层。
Web tier environments are meant for web applications - http/https request processing. You get one or more EC2 instances behind a load balancer. You can get other resources like database per your requirement. You can choose the platform you wish e.g. Ruby, Python, Java, Node.js, PHP, Docker.
Web层环境适用于Web应用程序 - http / https请求处理。您在负载均衡器后面获得一个或多个EC2实例。您可以根据需要获取数据库等其他资源。您可以选择您想要的平台，例如Ruby，Python，Java，Node.js，PHP，Docker。
Worker environments are meant for asynchronous message processing. When you create a worker environment you do not have a load balancer. All your EC2 instances are in an autoscaling group. All these instances are running a daemon which is polling a single SQS queue for messages. When a message is pulled by the daemon from the SQS queue, the daemon sends a HTTP Post request on localhost:80. You can configure the port but the important thing is that the daemon posts the message as an HTTP request on localhost. Your worker application is actually a web application that receives the post request and processes the message. After the message is successfully processed the worker daemon expects that your web application running on localhost returns a HTTP 200 OK response. The daemon then deletes the message from SQS queue. You can write your worker application for any platform just like standard web server applications - Ruby, Python, Java, Node.js, PHP, Docker.
工作线程环境用于异步消息处理。创建工作线程环境时，您没有负载均衡器。您的所有EC2实例都在自动缩放组中。所有这些实例都在运行守护程序，该守护程序轮询单个SQS队列以查找消息。当守护程序从SQS队列中提取消息时，守护程序会在localhost：80上发送HTTP Post请求。您可以配置端口，但重要的是守护程序将消息作为localhost上的HTTP请求发布。您的工作者应用程序实际上是一个Web应用程序，它接收发布请求并处理消息。成功处理消息后，worker守护程序期望在localhost上运行的Web应用程序返回HTTP 200 OK响应。守护进程然后从SQS队列中删除该消息。您可以为任何平台编写工作者应用程序，就像标准Web服务器应用程序一样 - Ruby，Python，Java，Node.js，PHP，Docker。
Based on my understanding of your usecase I would recommend creating two Elastic Beanstalk environments - one Standard and one Worker environment. The Standard web server receives HTTP requests and processes them synchronously. This environment puts the relevant data in an SQS queue. The second environment is a worker and the daemon running on this environment polls this SQS queue for messages. Your second environment is a web application that is NOT open to the internet. The worker daemon posts the messages as HTTP requests to your worker environment. Thus you can process long running workloads asynchronously using this second worker environment.
根据我对您的用例的理解，我建议创建两个Elastic Beanstalk环境 - 一个Standard环境和一个Worker环境。标准Web服务器接收HTTP请求并同步处理它们。此环境将相关数据放入SQS队列中。第二个环境是一个worker，在此环境中运行的守护程序会轮询此SQS队列以获取消息。您的第二个环境是一个不向互联网开放的Web应用程序。 worker守护程序将消息作为HTTP请求发布到您的工作环境。因此，您可以使用此第二个工作线程环境异步处理长时间运行的工作负载。
With worker environments you can use your own queues or Elastic Beanstalk can generate a queue for you. You can configure parameters like message visibility timeout, http connections based on your requirements or you can use the defaults.
使用工作线程环境，您可以使用自己的队列，或者Elastic Beanstalk可以为您生成队列。您可以根据需要配置消息可见性超时，http连接等参数，也可以使用默认值。
Below are some links that may be useful for you:
以下是一些可能对您有用的链接：
http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/
http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/
http://blogs.aws.amazon.com/application-management/post/Tx1Y8QSQRL1KQZC/Elastic-Beanstalk-Video-Tutorial-Worker-Tier
https://*.com/a/23942498/161628
https://*.com/a/23942498/161628
Does this meet your requirements? Please let me know if you have further questions.
这符合您的要求吗？如果您有其他问题，请告诉我。
Update
更新
You need to upload your source code at two places - once for the worker environment and once for the web server environment. If someone was starting from scratch then they might have two separate code bases. But I think in your case I think it should be perfectly fine to have a single code base shared between the two environments. Suppose your web request arrives at '/register', then the register() method in your application can post messages to an SQS queue and be done with the HTTP request. Now your worker environment will poll the SQS queue and post messages over HTTP on localhost to the URL '/async_register' which will invoke a method async_register() in your application and do the asynchronous processing. These two methods can live in the same source code bundle which can be shared by both the worker and web server environment. The code path taken by worker and web server will be different so that web server environments will invoke register() and worker environments will invoke the async_register() method.
您需要在两个位置上传源代码 - 一次用于工作环境，一次用于Web服务器环境。如果有人从头开始，那么他们可能有两个独立的代码库。但我认为在你的情况下，我认为在两个环境之间共享一个代码库应该是完全没问题的。假设您的Web请求到达'/ register'，那么应用程序中的register（）方法可以将消息发布到SQS队列并完成HTTP请求。现在，您的工作线程环境将轮询SQS队列，并通过localhost上的HTTP发送消息到URL'/ async_register'，它将在您的应用程序中调用方法async_register（）并执行异步处理。这两种方法可以存在于同一个源代码包中，可以由工作者和Web服务器环境共享。工作者和Web服务器采用的代码路径将不同，因此Web服务器环境将调用register（），而工作线程环境将调用async_register（）方法。
Another caveat is that HTTP requests sent by the worker daemon on localhost will contain an HTTP header - "User-Agent": "aws-sqsd/1.1". Read more here. So in your web application you can have a single listener to post requests on "/register" and depending on whether this header is present or not you invoke register() or async_register() methods internally.
另一个警告是，工作守护程序在localhost上发送的HTTP请求将包含HTTP标头 - “User-Agent”：“aws-sqsd / 1.1”。在这里阅读更多。因此，在您的Web应用程序中，您可以让一个侦听器在“/ register”上发布请求，并根据是否存在此标头，在内部调用register（）或async_register（）方法。
Also I think if you want to share the code base between the two environments you can upload the code base at only one place. Your environments are logically grouped into applications. So you can have a single application. You upload your source code to this application using the "CreateApplicationVersion" API call. Suppose you upload an application version with label 'v1'. You can now create a worker environment and a web server environment under the same application. When you create an environment you need to provide a version to deploy to your enviroment. In this case you can deploy v1 to both environments. So you will be sharing the same source code for both environments. When you have a new version "v2". You upload this version and then perform an update on both environments changing their version to "v2".
另外我想如果你想在两个环境之间共享代码库，你可以只在一个地方上传代码库。您的环境在逻辑上分组到应用程序中。所以你可以拥有一个应用程序。您使用“CreateApplicationVersion”API调用将源代码上载到此应用程序。假设您上传了标签为“v1”的应用程序版本。您现在可以在同一应用程序下创建工作线程环境和Web服务器环境。创建环境时，您需要提供一个版本以部署到您的环境中。在这种情况下，您可以将v1部署到两个环境。因此，您将为两种环境共享相同的源代码。当你有一个新版本“v2”。您上传此版本，然后在两个环境中将其版本更改为“v2”时执行更新。
The same version of the source code can be deployed to both environments. They will be running on different EC2 instances because one environment is dedicated for responding to web requests and one environment is dedicated for responding to asynchronous web requests (worker).
可以将相同版本的源代码部署到两个环境中。它们将在不同的EC2实例上运行，因为一个环境专用于响应Web请求，一个环境专用于响应异步Web请求（工作者）。