We are using a cluster of node.js servers to power our chat system.
我们正在使用一组node.js服务器来为我们的聊天系统供电。
We would like users to be able to see their previous conversations with any other given user and as such need to store chat logs.
我们希望用户能够看到他们之前与任何其他给定用户的对话,因此需要存储聊天记录。
Since these never need to be edited or queried in any way it does not make sense to store them in a database and as such we would like to simply keep them as simple json files stored on Amazon S3.
由于这些从不需要以任何方式进行编辑或查询,因此将它们存储在数据库中是没有意义的,因此我们希望将它们简单地保存为存储在Amazon S3上的简单json文件。
Using Amazons PHP api it is now possible to use standard fopen/fwrite commands making streaming to S3 possible. But in this case we need to do it with Node.js.
使用Amazons PHP api现在可以使用标准的fopen / fwrite命令使流式传输到S3成为可能。但在这种情况下,我们需要使用Node.js。
Essentially we would like to be able to open a stream when the users begin chatting, and append to the end of a chatlog file on S3, live while the users are chatting.
基本上我们希望能够在用户开始聊天时打开流,并在用户聊天时附加到S3上的聊天记录文件的末尾。
If this is not possible, what are our other options?
如果无法做到这一点,我们的其他选择是什么?
We have considered creating local writeStreams on the node.js servers, using them to essentially buffer the data, detect the end of a conversation and then upload/replace the file to S3. It just seems overly verbose and given the the node.js servers are within Amazons network it feels like a poor mans solution when clearly streaming to S3 is now a possibility in PHP.
我们考虑在node.js服务器上创建本地writeStream,使用它们来实质上缓冲数据,检测对话结束,然后将文件上传/替换为S3。它似乎过于冗长,并且鉴于node.js服务器位于Amazons网络中,感觉就像一个糟糕的勒芒解决方案,现在可以在PHP中清晰地流式传输到S3。
Thanks in advance
提前致谢
1 个解决方案
#1
1
Since these never need to be edited or queried in any way it does not make sense to store them in a database...
I don't agree with your logic here: Just because you only want to write once and read-many, doesn't mean a DB is not an appropriate solution. You could probably use S3, but given the brief description of your problem I would think a classic DB solution is going to serve you better in the long run. If you go the DB route, you have lots of well supported options within AWS (RDS-SQL Server, Postgres, MYSQL), DynamocDB and SimpleDB.
我不同意你的逻辑:只是因为你只想写一次和多次读,并不意味着数据库不是一个合适的解决方案。您可以使用S3,但鉴于您的问题的简要描述,我认为从长远来看,经典的数据库解决方案将为您提供更好的服务。如果您使用数据库路由,则AWS(RDS-SQL Server,Postgres,MYSQL),DynamocDB和SimpleDB中有许多支持良好的选项。
Even if you end up storing the chats on S3, its not unreasonable to think you may end up storing the metadata about each chat in a DB such as DynamoDB. How many chats are you going to store? If you end up with 100's of thousands or millions of seperate chat files, how will you navigate a user to the right one when they want it? This is a where a DB will come in handy, and then it begs the question, if you are going to use a DB to store the metadata, why not just store the content as well?
即使您最终在S3上存储聊天记录,认为您最终可能会在数据库(例如DynamoDB)中存储有关每个聊天的元数据也是不合理的。你要存多少次聊天?如果最终得到100个数千或数百万个单独的聊天文件,那么如何在用户需要时将用户导航到正确的用户?这是一个数据库将派上用场的地方,然后问题就是,如果你要使用数据库来存储元数据,为什么不只是存储内容呢?
#1
1
Since these never need to be edited or queried in any way it does not make sense to store them in a database...
I don't agree with your logic here: Just because you only want to write once and read-many, doesn't mean a DB is not an appropriate solution. You could probably use S3, but given the brief description of your problem I would think a classic DB solution is going to serve you better in the long run. If you go the DB route, you have lots of well supported options within AWS (RDS-SQL Server, Postgres, MYSQL), DynamocDB and SimpleDB.
我不同意你的逻辑:只是因为你只想写一次和多次读,并不意味着数据库不是一个合适的解决方案。您可以使用S3,但鉴于您的问题的简要描述,我认为从长远来看,经典的数据库解决方案将为您提供更好的服务。如果您使用数据库路由,则AWS(RDS-SQL Server,Postgres,MYSQL),DynamocDB和SimpleDB中有许多支持良好的选项。
Even if you end up storing the chats on S3, its not unreasonable to think you may end up storing the metadata about each chat in a DB such as DynamoDB. How many chats are you going to store? If you end up with 100's of thousands or millions of seperate chat files, how will you navigate a user to the right one when they want it? This is a where a DB will come in handy, and then it begs the question, if you are going to use a DB to store the metadata, why not just store the content as well?
即使您最终在S3上存储聊天记录,认为您最终可能会在数据库(例如DynamoDB)中存储有关每个聊天的元数据也是不合理的。你要存多少次聊天?如果最终得到100个数千或数百万个单独的聊天文件,那么如何在用户需要时将用户导航到正确的用户?这是一个数据库将派上用场的地方,然后问题就是,如果你要使用数据库来存储元数据,为什么不只是存储内容呢?