我扩展nodejs应用的最好方法是什么?

The basics

Right now a few of my friends and I are trying to develope a browser game made in nodejs. It's a multiplayer top-down shooter, and most of both the client-side and server-side code is in javascript. We have a good general direction that we'd like to go in, and we're having a lot of fun developing the game. One of our goals when making this game was to make it as hard as possible to cheat. Do do that, we have all of the game logic handled server-side. The client only sends their input the the server via web socket, and the server updates the client (also web socket) with what is happening in the game. Here's the start of our problem.

现在，我和几个朋友正在开发一个用node . js制作的浏览器游戏。它是一个多玩家自顶向下的射手，而且大多数客户端和服务器端的代码都是javascript的。我们有一个好的总体方向，我们想要进入，我们有很多乐趣开发这个游戏。我们做这个游戏的目标之一就是让作弊变得尽可能的困难。做到这一点，我们已经处理了服务器端所有的游戏逻辑。客户端只通过web套接字向服务器发送他们的输入，服务器用游戏中发生的事情更新客户端(也是web套接字)。这是问题的开始。

All of the server side math is getting pretty hefty, and we're finding that we need to scale in some way to handle anything more than 10 players (we want to be able to host many more). At first we had figured that we could just scale vertically as we needed to, but since nodejs is single threaded, is can only take advantage of one core. This means that getting a beefier server won't help that problem. Our only solution is to scale horizontally.

所有的服务器端计算都变得非常复杂，我们发现我们需要以某种方式扩展来处理超过10个玩家(我们希望能够托管更多)。一开始我们认为可以按需要垂直伸缩，但是由于nodejs是单线程的，is只能利用一个核心。这意味着获得一个更强大的服务器不会解决这个问题。我们唯一的解决方案是水平伸缩。

Why we're asking here

We haven't been able to find any good examples of how to scale out a nodejs game. Our use case is pretty particular, and while we've done our best to do this by ourselves, we could really benefit from outside opinions and advice

我们还没有找到任何关于如何扩展node . js游戏的好例子。我们的用例是非常特别的，虽然我们已经尽了最大的努力来做到这一点，但是我们可以从外界的意见和建议中获益

Details

We've already put a LOT of thought into how to solve this problem. We've been working on it for over a week. Here's what we have put together so far:

我们已经花了很多心思在如何解决这个问题上。我们已经做了一个多星期了。以下是我们到目前为止收集到的信息:

Four types of servers

We're splitting tasks into 4 different 'types' of servers. Each one will have a specific task it completes.

我们将任务分成4种不同的“类型”服务器。每个人都会有一个特定的任务完成。

The proxy server

The proxy server would sit at the front of the entire stack, and be the only server directly accessible from the internet (there could potentially be more of these). It would have haproxy on it, and it would route all connections to the web servers. We chose haproxy because of its rich feature set, reliability, and nearly unbeatable speed.

代理服务器将位于整个堆栈的前面，并且是唯一可以从internet直接访问的服务器(可能有更多这样的服务器)。它会有haproxy，它会将所有连接路由到web服务器。我们之所以选择haproxy，是因为它具有丰富的特性集、可靠性和几乎不可战胜的速度。

The web server

The web server would receive the web-requests, and serve all web-pages. They would also handle lobby creation/management and game creation/management. To do this, they would tell the game servers what lobbies it has, what users are in that lobby, and info about the game they're going to play. The web servers would then update the game servers about user input, and the game server would update the web servers (who would then update the clients) of what's happening in the game. The web servers would use TCP sockets to communicate with the game servers about any type of management, and they would use UDP sockets when communicating about game updates. This would all be done with nodejs.

web服务器将接收web请求，并服务于所有web页面。他们还将处理游说创造/管理和游戏创建/管理。要做到这一点，他们会告诉游戏服务器它有什么大厅，用户在大厅里是什么，以及他们将要玩的游戏的信息。然后，web服务器会更新游戏服务器关于用户输入的信息，而游戏服务器会更新游戏中发生的事情的web服务器(然后由谁更新客户端)。web服务器将使用TCP套接字与游戏服务器通信任何类型的管理，并且在通信游戏更新时使用UDP套接字。这一切都要用node . js来完成。

The game server

The game server would handle all the game math and variable updates about the game. The game servers also communicate with the db servers to record cool stats about players in game. This would be done with nodejs.

游戏服务器将处理游戏的所有数学和变量更新。游戏服务器还与db服务器通信，以记录游戏中玩家的酷数据。这将由nodejs完成。

The db server

The db server would host the database. This part actually turned out to be the easiest since we found rethinkdb, the coolest db ever. This scales easily, and oddly enough, turned out to be the easiest part of scaling our application.

db服务器将托管数据库。这部分实际上是最容易的，因为我们发现了rethinkdb，这是最酷的db。这很容易扩展，而且非常奇怪，结果是扩展应用程序最容易的部分。

Some other details

If you're having trouble getting your head around our whole getup, look at this, it's a semi-accurate chart of how we think we'll scale.

如果你在我们的整个行动中遇到困难，看看这个，它是一个半精确的图表，我们认为我们将如何衡量。

If you're just curious, or think it might be helpful to look at our game, it's currently hosted in it's un-scaled state here.

如果你只是好奇，或者认为看看我们的游戏可能会有帮助，它目前托管在它的未缩放的状态。

Some things we don't want

• We don't want to use the cluster module of nodejs. It isn't stable (said here), and it doesn't scale to other servers, only other processors. We'd like to just take the leap to horizontal scaling.
• 我们不想使用nodejs的集群模块。它不稳定(这里说的)，并且不扩展到其他服务器，只扩展到其他处理器。我们想直接跳到水平缩放。

Our question, summed up

We hope we're going in the right direction, and we've done our homework, but we're not certain. We could certainly take a few tips on how to do this the right way.

我们希望我们的方向是正确的，我们已经做了功课，但我们不确定。我们当然可以采取一些技巧，如何以正确的方式做到这一点。

Thanks

I realize that this is a pretty long question, and making a well thought out answer will not be easy, but I would really appreciate it.

我意识到这是一个很长的问题，想好答案并不容易，但我真的很感激。

Thanks!!

谢谢! !

2 个解决方案

#1

Following my spontaneous thoughts on your case:

以下是我对你的案例的自发思考:

Multicore usage

node.js can scale with multiple cores as well. How, you can read for example here (or just think about it: You have one thread/process running on one core, what do you need to use multiple cores? Multiple threads or multiple processes. Push work from main thread to other threads or processes and you are done).

节点。js也可以使用多个内核进行扩展。例如，您可以在这里阅读(或者想一想:在一个内核上运行一个线程/进程，需要使用多个内核吗?多个线程或多个进程。将工作从主线程推到其他线程或进程，您就完成了)。

I personally would say it is childish to develop an application, which does not make use of multiple cores. If you make use of some background processes, ok, but if you until now only do work in the node.js main event loop, you should definitely invest some time to make the app scalable over cores.

我个人认为，开发一个不使用多核的应用程序很幼稚。如果您使用了一些后台进程，但是如果您直到现在只在节点中工作。js主事件循环，您肯定应该花一些时间使应用程序在核心上可伸缩。

Implementing something like IPC is not that easy by the way. You can do, but if your case is complicated maybe you are good to go with the cluster module. This is obviously not your favorite, but just because something is called "experimental" it does not mean it's trashy. Just give it a try, maybe you can even fix some bugs of the module on the way. It's most likely better to use some broadly used software for complex problems, than invent a new wheel.

顺便说一句，实现IPC之类的东西并不容易。您可以这样做，但是如果您的情况比较复杂，那么您最好使用集群模块。这显然不是你的最爱，但仅仅因为被称为“实验”的东西并不意味着它是无用的。试一试，也许你可以在路上修复一些模块的错误。在复杂的问题上使用一些广泛使用的软件比发明一个新的*要好。

You should also (if you do not already) think about (wise) usage of nextTick functionality. This allows the main event loop to pause some cpu intensive task and perform other work in the meanwhile. You can read about it for example here.

您还应该(如果您还没有)考虑下nextTick功能的(明智的)用法。这允许主事件循环暂停一些cpu密集型任务，同时执行其他工作。你可以在这里读到。

General thoughts on computations

You should definitely take a very close look at your algorithms of the game engine. You already noticed that this is your bottleneck right now and actually computations are the most critical part of mostly every game. Scaling does solve this problem in one way, but scaling introduces other problems. Also you cannot throw "scaling" as problem solver on everything and expect every problem to disappear.

你一定要仔细看看你的游戏引擎算法。你已经注意到这是你现在的瓶颈，实际上计算是大部分游戏中最关键的部分。扩展确实以一种方式解决了这个问题，但是扩展引入了其他问题。同样，你也不能把“扩展”作为问题解决者，期望每个问题都消失。

Your best bet is to make your game code elegant and fast. Think about how to solve problems efficiently. If you cannot solve something in Javascript efficiently, but the problem can easily be extracted, why not write a little C component instead? This counts as a separate process as well, which reduces load on your main node.js event loop.

你最好的办法是使你的游戏代码优雅而快速。思考如何有效地解决问题。如果您不能有效地解决Javascript中的问题，但是问题可以很容易地提取出来，为什么不编写一个小的C组件呢?这也算作一个单独的进程，减少了主节点的负载。js事件循环。

Proxy?

Personally I do not see the advantage of the proxy level right now. You do not seem to expect large amount of users, you therefore won't need to solve problems like CDN solves or whatever... it's okay to think about it, but I would not invest much time there right now.

就我个人而言，我现在还看不到代理级别的优势。您似乎并不期望有大量的用户，因此您不需要解决诸如CDN之类的问题……可以考虑一下，但我现在不会在那里花太多时间。

Technically there is a high chance your webserver software provides proxy functionality anyway. So it is ok to have it on the paper, but I would not plan with dedicated hardware right now.

从技术上讲，web服务器软件提供代理功能的可能性很大。所以把它写在纸上是可以的，但是我现在不打算用专用的硬件。

Epilogue

The rest seems more or less fine to me.

其余的在我看来都差不多。

#2

Little late to the game, but take a look here: http://goldfirestudios.com/blog/136/Horizontally-Scaling-Node.js-and-WebSockets-with-Redis

这款游戏有点晚了，但请看看这里:http://goldfirestudios.com/blog/136/horizontally-scaling-node.jand - websockets-with-redis。

You did not mention anything to do with memory management. As you know, nodejs doesn't share its memory with other processes, so an in-memory database is a must if you want to scale. (Redis, Memcache, etc). You need to setup a publisher & subscriber event on each node to accept incoming requests from redis. This way, you can scale up x nilo amount of servers (infront of your HAProxy) and utilize the data piped from redis.

您没有提到任何与内存管理有关的内容。正如您所知道的，nodejs不会与其他进程共享内存，因此如果您希望扩展内存数据库，那么内存中数据库是必需的。(复述、Memcache等)。您需要在每个节点上设置发布者和订阅者事件，以接受来自redis的传入请求。通过这种方式，您可以扩展x nilo数量的服务器(在HAProxy的前面)，并利用来自redis的数据管道。

There is also this node addon: http://blog.varunajayasiri.com/shared-memory-with-nodejs That lets you share memory between processes, but only works under Linux. This will help if you don't want to send data across local processes all the time or have to deal with nodes ipc api.

还有这个节点addon: http://blog.varunajayasiri.com/shared-memory-with-nodejs，它允许您在进程之间共享内存，但只能在Linux下工作。如果您不希望一直跨本地进程发送数据，或者必须处理节点ipc api，那么这将有所帮助。

You can also fork child processes within node for a new v8 isolate to help with expensive cpu bound tasks. For example, players can kill monsters and obtain quite a bit of loot within my action rpg game. I have a child process called LootGenerater, and basically whenever a player kills a monster it sends the game id, mob_id, and user_id to the process via the default IPC api .send. Once the child process receives it, it iterates over the large loot table and manages the items (stores to redis, or whatever) and pipes it back.

您还可以在node中为新的v8隔离程序提供子进程，以帮助处理昂贵的cpu绑定任务。例如，玩家可以杀死怪物并在我的动作rpg游戏中获得相当多的战利品。我有一个叫做LootGenerater的子进程，基本上只要玩家杀死一个怪物，它就会通过默认的IPC api .send将游戏id、mob_id和user_id发送给进程。一旦子进程接收到它，它将遍历大型loot表并管理条目(存储到redis或其他)并将其返回。

This helps free up the event loop greatly, and just one idea I can think of to help you scale. But most importantly you will want to use an in-memory database system and make sure your game code architecture is designed around whatever database system you use. Don't make the mistake I did by now having to re-write everything :)

这有助于极大地释放事件循环，我能想到的只是一个可以帮助您扩展的想法。但最重要的是，您将希望使用内存中的数据库系统，并确保您的游戏代码架构是围绕您使用的任何数据库系统设计的。不要犯我现在不得不重写的错误:

Hope this helps!

希望这可以帮助!

Note: If you do decide to go with Memcache, you will need to utilize another pub/sub system.

注意:如果您决定使用Memcache，您将需要使用另一个pub/sub系统。