I recently experienced a flood of traffic on a Facebook app I created (mostly for the sake of education, not with any intention of marketing)
我最近在Facebook上创建的一个应用程序上遭遇了大量的流量(主要是为了教育,而不是为了营销)
Needless to say, I did not think about scalability when I created the app. I'm now in a position where my meager virtual server hosted by MediaTemple isn't cutting it at all, and it's really coming down to raw I/O of the machine. Since this project has been so educating to me so far, I figured I'd take this as an opportunity to understand the Amazon EC2 platform.
不用说,当我创建这个应用程序时,我并没有考虑到可伸缩性。我现在处于一个位置,在这个位置,我的MediaTemple托管的虚拟服务器并没有削减它,它实际上是在机器的原始I/O上。由于这个项目到目前为止对我的教育程度非常高,我认为我可以借此机会了解Amazon EC2平台。
The app itself is created in PHP (using Zend Framework) with a MySQL backend. I use application caching wherever possible with memcached. I've spent the weekend playing around with EC2, spinning up instances, installing the packages I want, and mounting an EBS volume to an instance.
应用程序本身是用PHP(使用Zend Framework)和MySQL后端创建的。我尽可能地使用memcached应用程序缓存。我花了整个周末在EC2上玩,转个实例,安装我想要的软件包,并将一个EBS卷安装到一个实例中。
But what's the next logical step that is going to yield good results for scalability? Do I fire up an AMI instance for the MySQL and one for the Apache service? Or do I just replicate the instances out as many times as I need them and then do some sort of load balancing on the front end? Ideally, I'd like to have a centralized database because I do aggregate statistics across all database rows, however, this is not a hard requirement (there are probably some application specific solutions I could come up with to work around this)
但是,为可扩展性产生良好结果的下一个合乎逻辑的步骤是什么呢?我是否为MySQL启动一个AMI实例,为Apache服务启动一个AMI实例?或者,我只是在需要的时候多次复制实例,然后在前端执行某种负载平衡?理想情况下,我希望有一个集中式的数据库,因为我在所有数据库行中聚合统计数据,但是,这并不是一个困难的需求(我可能会提出一些特定于应用程序的解决方案来解决这个问题)
I know this is probably not a straight forward answer, so opinions and suggestions are welcome.
我知道这可能不是一个直截了当的回答,所以欢迎大家提出意见和建议。
2 个解决方案
#1
14
So many questions - all of them good though.
有这么多问题,但都很好。
In terms of scaling, you've a few options.
在缩放方面,您有一些选择。
The first is to start with a single box. You can scale upwards - with a more powerful box. EC2 have various sized instances. This involves a server migration each time you want a bigger box.
首先,从一个盒子开始。你可以用一个更强大的盒子来放大。EC2有各种大小的实例。这涉及到每次您想要一个更大的盒子时的服务器迁移。
Easier is to add servers. You can start with a single instance for Apache & MySQL. Then when traffic increases, create a separate instance for MySQL and point your application to this new instance. This creates a nice layer between application and database. It sounds like this is a good starting point based on your traffic.
更容易的是添加服务器。您可以从一个Apache & MySQL实例开始。然后,当流量增加时,为MySQL创建一个单独的实例,并将应用程序指向这个新实例。这在应用程序和数据库之间创建了一个很好的层。听起来这是一个很好的起点,基于你的流量。
Next you'll probably need more application power (web servers) or more database power (MySQL cluster etc.). You can have your DNS records pointing to a couple of front boxes running some load balancing software (try Pound). These load balancing servers distribute requests to your webservers. EC2 has Elastic Load Balancing which is an alternative to managing this yourself, and is probably easier - I haven't used it personally.
接下来,您可能需要更多的应用程序能力(web服务器)或更多的数据库能力(MySQL集群等)。您可以让您的DNS记录指向运行负载平衡软件的几个前箱(请尝试Pound)。这些负载平衡服务器将请求分发到web服务器。EC2有弹性负载平衡,这是管理它的一种替代方法,而且可能更简单——我还没有亲自使用它。
Something else to be aware of - EC2 has no persistent storage. You have to manage persistent data yourself using the Elastic Block Store. This guide is an excellent tutorial on how to do this, with automated backups.
需要注意的是——EC2没有持久存储。您必须使用弹性块存储来管理持久数据。本指南是一个关于如何使用自动备份来实现这一点的优秀教程。
I recommend that you purchase some reserved instances if you decide EC2 is the way forward. You'll save yourself about 50% over 3 years!
如果您认为EC2是未来的发展方向,我建议您购买一些保留实例。你可以在3年内节省50%的费用!
Finally, you may be interested in services like RightScale which offer management services at a cost. There are other providers available.
最后,您可能对RightScale这样的服务感兴趣,该公司提供的管理服务是收费的。还有其他可用的供应商。
#2
10
First step is to separate concerns. I'd split off with a separate MySQL server and possibly a dedicated memcached box, depending on how high your load is there. Then I'd monitor memory and CPU usage on each box and see where you can optimize where possible. This can be done with spinning off new Media Temple boxes. I'd also suggest Slicehost for a cheaper, more developer-friendly alternative.
第一步是分离关注点。我将独立的MySQL服务器和专用的memcached框分开,这取决于您的负载有多高。然后,我将监视每个框上的内存和CPU使用情况,看看在什么地方可以进行优化。这可以通过分离新媒体圣殿盒子来实现。我还建议使用Slicehost作为一个更便宜、更有利于开发人员的替代方案。
Some more low-budget PHP deployment optimizations:
一些更低成本的PHP部署优化:
- Using a more efficient web server like nginx to handle static file serving and then reverse proxy app requests to a separate Apache instance
- 使用更高效的web服务器(如nginx)来处理静态文件服务,然后将代理应用程序请求转到一个单独的Apache实例。
- Implement PHP with FastCGI on top of nginx using something like PHP-FPM, getting rid of Apache entirely. This may be a great alternative if your Apache needs don't extend far beyond
mod_rewrite
and simpler Apache modules. - 在nginx上使用FastCGI实现PHP,使用PHP- fpm之类的工具,完全摆脱Apache。如果您的Apache需要的不仅仅是mod_rewrite和更简单的Apache模块,那么这可能是一个很好的选择。
If you prefer a more high-level, do-it-yourself approach, you may want to check out Scalr (code at Google Code). It's worth watching the video on their web site. It facilities a scalable hosting environment using Amazon EC2. The technology is open source, so you can download it and implement it yourself on your own management server. (Your Media Temple box, perhaps?) Scalr has pre-built AMIs (EC2 appliances) available for some common use cases.
如果您喜欢更高级的、自己动手的方法,那么您可能需要检查Scalr(谷歌代码上的代码)。在他们的网站上看这个视频是值得的。它使用Amazon EC2提供可伸缩的托管环境。该技术是开源的,因此您可以下载它并在自己的管理服务器上实现它。(也许是你的媒体神殿吧?)Scalr已经为一些常见的用例预先构建了ami (EC2设备)。
- web: Utilizes nginx and its many capabilities: software load balancing, static file serving, etc. You'd probably only have one of these, and it would probably implement some sort of connection to Amazon's EBS, or persistent storage solution, as mentioned by dcaunt.
- web:使用nginx及其许多功能:软件负载平衡、静态文件服务等。您可能只有其中之一,它可能实现与Amazon的EBS的某种连接,或dcaunt提到的持久存储解决方案。
- app: An application server with Apache and PHP. You'd probably have many of these, and they'd get created automatically if more load needed to be handled. This type of server would hold copies of your ZF app.
- 应用程序:带有Apache和PHP的应用服务器。您可能有很多这样的功能,如果需要处理更多的负载,它们会自动创建。这种类型的服务器将保存ZF应用程序的副本。
- db: A database server with MySQL. Again, you'd probably have many of these, and more slave instances would get created automatically if more load needed to be handled.
- db: MySQL数据库服务器。同样,您可能有许多这样的实例,如果需要处理更多的负载,将自动创建更多的从属实例。
- memcached: A dedicated memcached server you can use to have centralized caching, session management, et cetera across all your app instances.
- memcached:一个专用的memcached服务器,您可以使用它在所有应用程序实例中进行集中缓存、会话管理等等。
The Scalr option will probably take some more configuration changes, but if you feel your scaling needs accelerating quickly it may be worth the time and effort.
Scalr选项可能需要更多的配置更改,但是如果您觉得伸缩需要加快,那么它可能值得花费时间和精力。
#1
14
So many questions - all of them good though.
有这么多问题,但都很好。
In terms of scaling, you've a few options.
在缩放方面,您有一些选择。
The first is to start with a single box. You can scale upwards - with a more powerful box. EC2 have various sized instances. This involves a server migration each time you want a bigger box.
首先,从一个盒子开始。你可以用一个更强大的盒子来放大。EC2有各种大小的实例。这涉及到每次您想要一个更大的盒子时的服务器迁移。
Easier is to add servers. You can start with a single instance for Apache & MySQL. Then when traffic increases, create a separate instance for MySQL and point your application to this new instance. This creates a nice layer between application and database. It sounds like this is a good starting point based on your traffic.
更容易的是添加服务器。您可以从一个Apache & MySQL实例开始。然后,当流量增加时,为MySQL创建一个单独的实例,并将应用程序指向这个新实例。这在应用程序和数据库之间创建了一个很好的层。听起来这是一个很好的起点,基于你的流量。
Next you'll probably need more application power (web servers) or more database power (MySQL cluster etc.). You can have your DNS records pointing to a couple of front boxes running some load balancing software (try Pound). These load balancing servers distribute requests to your webservers. EC2 has Elastic Load Balancing which is an alternative to managing this yourself, and is probably easier - I haven't used it personally.
接下来,您可能需要更多的应用程序能力(web服务器)或更多的数据库能力(MySQL集群等)。您可以让您的DNS记录指向运行负载平衡软件的几个前箱(请尝试Pound)。这些负载平衡服务器将请求分发到web服务器。EC2有弹性负载平衡,这是管理它的一种替代方法,而且可能更简单——我还没有亲自使用它。
Something else to be aware of - EC2 has no persistent storage. You have to manage persistent data yourself using the Elastic Block Store. This guide is an excellent tutorial on how to do this, with automated backups.
需要注意的是——EC2没有持久存储。您必须使用弹性块存储来管理持久数据。本指南是一个关于如何使用自动备份来实现这一点的优秀教程。
I recommend that you purchase some reserved instances if you decide EC2 is the way forward. You'll save yourself about 50% over 3 years!
如果您认为EC2是未来的发展方向,我建议您购买一些保留实例。你可以在3年内节省50%的费用!
Finally, you may be interested in services like RightScale which offer management services at a cost. There are other providers available.
最后,您可能对RightScale这样的服务感兴趣,该公司提供的管理服务是收费的。还有其他可用的供应商。
#2
10
First step is to separate concerns. I'd split off with a separate MySQL server and possibly a dedicated memcached box, depending on how high your load is there. Then I'd monitor memory and CPU usage on each box and see where you can optimize where possible. This can be done with spinning off new Media Temple boxes. I'd also suggest Slicehost for a cheaper, more developer-friendly alternative.
第一步是分离关注点。我将独立的MySQL服务器和专用的memcached框分开,这取决于您的负载有多高。然后,我将监视每个框上的内存和CPU使用情况,看看在什么地方可以进行优化。这可以通过分离新媒体圣殿盒子来实现。我还建议使用Slicehost作为一个更便宜、更有利于开发人员的替代方案。
Some more low-budget PHP deployment optimizations:
一些更低成本的PHP部署优化:
- Using a more efficient web server like nginx to handle static file serving and then reverse proxy app requests to a separate Apache instance
- 使用更高效的web服务器(如nginx)来处理静态文件服务,然后将代理应用程序请求转到一个单独的Apache实例。
- Implement PHP with FastCGI on top of nginx using something like PHP-FPM, getting rid of Apache entirely. This may be a great alternative if your Apache needs don't extend far beyond
mod_rewrite
and simpler Apache modules. - 在nginx上使用FastCGI实现PHP,使用PHP- fpm之类的工具,完全摆脱Apache。如果您的Apache需要的不仅仅是mod_rewrite和更简单的Apache模块,那么这可能是一个很好的选择。
If you prefer a more high-level, do-it-yourself approach, you may want to check out Scalr (code at Google Code). It's worth watching the video on their web site. It facilities a scalable hosting environment using Amazon EC2. The technology is open source, so you can download it and implement it yourself on your own management server. (Your Media Temple box, perhaps?) Scalr has pre-built AMIs (EC2 appliances) available for some common use cases.
如果您喜欢更高级的、自己动手的方法,那么您可能需要检查Scalr(谷歌代码上的代码)。在他们的网站上看这个视频是值得的。它使用Amazon EC2提供可伸缩的托管环境。该技术是开源的,因此您可以下载它并在自己的管理服务器上实现它。(也许是你的媒体神殿吧?)Scalr已经为一些常见的用例预先构建了ami (EC2设备)。
- web: Utilizes nginx and its many capabilities: software load balancing, static file serving, etc. You'd probably only have one of these, and it would probably implement some sort of connection to Amazon's EBS, or persistent storage solution, as mentioned by dcaunt.
- web:使用nginx及其许多功能:软件负载平衡、静态文件服务等。您可能只有其中之一,它可能实现与Amazon的EBS的某种连接,或dcaunt提到的持久存储解决方案。
- app: An application server with Apache and PHP. You'd probably have many of these, and they'd get created automatically if more load needed to be handled. This type of server would hold copies of your ZF app.
- 应用程序:带有Apache和PHP的应用服务器。您可能有很多这样的功能,如果需要处理更多的负载,它们会自动创建。这种类型的服务器将保存ZF应用程序的副本。
- db: A database server with MySQL. Again, you'd probably have many of these, and more slave instances would get created automatically if more load needed to be handled.
- db: MySQL数据库服务器。同样,您可能有许多这样的实例,如果需要处理更多的负载,将自动创建更多的从属实例。
- memcached: A dedicated memcached server you can use to have centralized caching, session management, et cetera across all your app instances.
- memcached:一个专用的memcached服务器,您可以使用它在所有应用程序实例中进行集中缓存、会话管理等等。
The Scalr option will probably take some more configuration changes, but if you feel your scaling needs accelerating quickly it may be worth the time and effort.
Scalr选项可能需要更多的配置更改,但是如果您觉得伸缩需要加快,那么它可能值得花费时间和精力。