通过Amazon SQS进行漏斗式数据库写入

So assume im building netflix and I want to log each view by the userID and the movie ID

假设我在构建netflix，我想用用户ID和电影ID来记录每个视图。

so its like viewID , userID, timestamp,

比如viewID, userID, timestamp，

however inorder to scale this, assume were getting 1000 views a second (just to be crazy). Would it make sense to queue these views to SQS and then our queue readers can un-queue one by one and write it to the mysql database. This way the database isnt overloaded with write requests.

然而，为了扩展这个范围，假设每秒有1000个视图(这简直太疯狂了)。将这些视图排队到SQS中，然后我们的队列读取器可以逐个取消队列，并将其写到mysql数据库中，这样做是否有意义?这样，数据库就不会被写请求重载。

Does this look like it would work?

这看起来可行吗?

1 个解决方案

#1

Faisal,

费萨尔,

This is a reasonable architecture; however, you should know that writing to SQS is going to be many times slower than writing to something like RabbitMQ (or any local) message queue.

这是一个合理的架构;但是，您应该知道，编写到SQS的速度要比写入RabbitMQ(或任何本地)消息队列慢很多倍。

That being said, starting with SQS wouldn't be a bad idea since it is easy to use and debug.

也就是说，从SQS开始并不是一个坏主意，因为它很容易使用和调试。

Additionally, you may want to investigate MongoDB for logging...check out the following references:

此外，您可能想要调查MongoDB的日志记录……查看以下参考文献:

MongoDB is Fantastic for Logging

MongoDB非常适合日志记录

http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-logging

Capped Collections

限制集合

http://blog.mongodb.org/post/116405435/capped-collections

Using MongoDB for Real-time Analytics

使用MongoDB进行实时分析

http://blog.mongodb.org/post/171353301/using-mongodb-for-real-time-analytics

#1