I have a social network
我有一个社交网络
-
The users table is around 60,000 rows
users表大约有60,000行
-
The friends table is around 1 million rows (used to determine who is your
friend)朋友表约有100万行(用于确定谁是你的朋友)
I am wanting to do a friend feed, wall, whatever you like to call it, it will show things like user status post (twitter type posts), it will show a few different items but for the start it will just be friend status and maybe blog post.
我想做一个朋友饲料,墙,无论你喜欢什么叫它,它会显示用户状态帖子(推特类型的帖子)之类的东西,它会显示一些不同的项目但是一开始它只是朋友状态和也许博客文章。
Basicly you will only see content published with a user ID that is in your friend list.
基本上,您只会看到使用您朋友列表中的用户ID发布的内容。
I have been trying to come up with the best way and haven't gotten very far but here is my latest idea.
我一直试图想出最好的方法而且没有走得太远,但这是我最新的想法。
Currently to build this feed, I have to
目前要建立这个饲料,我必须
-
Get the list of friend ID's on the large friend table
获取大朋友桌上的朋友ID列表
-
Get the stream data from the friend ids from the above result
从上面的结果中获取朋友ID中的流数据
-
JOIN the user table to get the publishers picture URL and username
加入用户表以获取发布者图片URL和用户名
-
Then JOIN the comments table to get comments posted on the feed items
然后加入评论表以获取在Feed项目上发布的评论
That is one big task to build that feed
这是构建该Feed的一项重大任务
I have 3 ideas so far, this is where your help can come in.
到目前为止,我有3个想法,这是你的帮助可以进来的地方。
Memcache Option:
Memcache选项:
- Use memcache and cache a users friendlist as an array when the user logs into the site, also when the user approves a new friend request for a friend to be added to there list, it would rebuild there cache.
- 当用户登录到站点时,使用memcache并将用户friendlist缓存为数组,当用户批准新朋友请求将某个朋友添加到该列表时,它将重建那里的缓存。
- In addition to just getting there friends I could save there friends picture URL and username, this would speed up things again by eliminating this query when building the friend feed.
- 除了刚刚到那里的朋友,我可以保存朋友图片网址和用户名,这将通过在构建朋友Feed时消除此查询来再次加快速度。
File cache Option:
文件缓存选项:
-
Do the same as the memcache option does but save this data as an array to a cache file instead of memory, then include this cache file into the page.
执行与memcache选项相同但将此数据作为数组保存到缓存文件而不是内存,然后将此缓存文件包含在页面中。
-
I am not sure which is the best method for performance I understand memcache stores everything in memory so friends that have like 20,000 friends that could use a lot of memory and a file cache would only put it in memory when the users needs it if I am correct. Also if I did the file method, when a user logs out of the site, I would delete there cache file so the cache folder would never be too large of files
我不确定哪种是性能最好的方法我理解memcache将所有内容存储在内存中所以有两万个朋友可以使用大量内存和文件缓存的朋友只有在用户需要时才将它放在内存中如果我是正确。另外,如果我使用文件方法,当用户退出网站时,我会删除缓存文件,因此缓存文件夹永远不会太大的文件
Session cache Option:
会话缓存选项:
- Same as file cache above, I just realized that session data is saved into a file so wouldn't that make it capable of being a cache?
- 与上面的文件缓存相同,我只是意识到会话数据被保存到文件中,这样就不会使它成为缓存吗?
Please give me your opinions or any advice or info you have on this as I don't have much knowledge of caching, I have read a lot but sometimes other peoples ideas help a lot
请给我你的意见或任何有关此的建议或信息,因为我对缓存知之甚少,我已经阅读了很多,但有时其他人的想法有很多帮助
3 个解决方案
#1
29
Memcache is your best bet for a lot of reasons:
由于很多原因,Memcache是你最好的选择:
- It's REALLY fast - Everything's in memory, and it's highly optimized for situations just like yours (and caching in general :)
- 它真的很快 - 一切都在内存中,并且它非常适合像你一样的情况(和一般的缓存:)
- It's distributed - This means that if you have multiple web / app servers running, they can all access the same cache
- 它是分布式的 - 这意味着如果您运行多个Web / app服务器,它们都可以访问相同的缓存
- You can pool multiple servers for memcache - If you've got a few servers that are relatively underutilized (or several dedicated cache servers), you can pool them all together into one big cache
- 您可以为memcache汇集多个服务器 - 如果您有一些相对未充分利用的服务器(或几个专用缓存服务器),您可以将它们全部集中到一个大缓存中
- It's super-scalable (for the reasons mentioned prior)
- 它具有超级可扩展性(前面提到的原因)
- It's got great PHP support - The PECL package for memcache was recently updated with a lot of new goodness
- 它有很好的PHP支持 - 用于memcache的PECL包最近更新了很多新的优点
- You can even store your user sessions in memcache - just set it up in your php.ini file. This is much faster than storing sessions in databases, and allows your sessions to persist across multiple web hosts (if you're in a load balanced situation)... this will also give your site a bit of a performance boost as there's no need to hit the filesystem / database for session info on every request.
- 您甚至可以将用户会话存储在memcache中 - 只需将其设置在php.ini文件中即可。这比在数据库中存储会话要快得多,并且允许您的会话在多个Web主机上保持不变(如果您处于负载平衡状态)...这也将为您的站点提供一点性能提升,因为不需要在每个请求上点击文件系统/数据库以获取会话信息。
... and many more ;)
... 还有很多 ;)
As to some of your concerns about memory footprint of individual cached items you've got a few options. My initial thought is to just give it a whirl, see how big these cache items really get (you can find several open-source things to monitor the actual cache usage, such as cacti). I think they'll be smaller than you'd think.
至于您对单个缓存项目的内存占用的一些担忧,您有一些选择。我最初的想法是给它一个旋转,看看这些缓存项目真正有多大(你可以找到几个开源的东西来监控实际的缓存使用情况,比如cacti)。我认为它们会比你想象的要小。
If they're not, I'd suggest re-thinking your cache strategy as far as what you actually cache, for how long, etc. Maybe you could build the feed from several things already in the cache (i.e. cache individual user data, and then build the feed for a person from all those individual items in cache). There are a lot of good articles out there on that front, just search 'em out :)
如果他们不是,我建议你重新思考你的缓存策略,直到实际缓存,持续多长时间等等。也许你可以从缓存中的几个东西构建feed(即缓存单个用户数据,然后从缓存中的所有单个项目构建一个人的订阅源。在这方面有很多好文章,只是搜索出来:)
#2
1
The default maximum object size that is allowed in Memcache is 1MB.
Memcache中允许的默认最大对象大小为1MB。
@jsaondavis : "session data is saved into a file".
@jsaondavis:“会话数据保存到文件中”。
Your above statement is wrong. Session can be configured to store in database. Default session hadndler is file.
您的上述陈述是错误的。会话可以配置为存储在数据库中。默认会话hadndler是文件。
#3
1
Redis would be a good solution:
Redis将是一个很好的解决方案:
Here is a thread on the Redis Vs. Memcached. Sounds like Redis has 512mb storage instead of the 1mb limit... QUITE a bit different :)
这是Redis Vs上的一个主题。 Memcached的。听起来像Redis有512mb的存储而不是1mb的限制......有点不同:)
Memcached与Redis?
#1
29
Memcache is your best bet for a lot of reasons:
由于很多原因,Memcache是你最好的选择:
- It's REALLY fast - Everything's in memory, and it's highly optimized for situations just like yours (and caching in general :)
- 它真的很快 - 一切都在内存中,并且它非常适合像你一样的情况(和一般的缓存:)
- It's distributed - This means that if you have multiple web / app servers running, they can all access the same cache
- 它是分布式的 - 这意味着如果您运行多个Web / app服务器,它们都可以访问相同的缓存
- You can pool multiple servers for memcache - If you've got a few servers that are relatively underutilized (or several dedicated cache servers), you can pool them all together into one big cache
- 您可以为memcache汇集多个服务器 - 如果您有一些相对未充分利用的服务器(或几个专用缓存服务器),您可以将它们全部集中到一个大缓存中
- It's super-scalable (for the reasons mentioned prior)
- 它具有超级可扩展性(前面提到的原因)
- It's got great PHP support - The PECL package for memcache was recently updated with a lot of new goodness
- 它有很好的PHP支持 - 用于memcache的PECL包最近更新了很多新的优点
- You can even store your user sessions in memcache - just set it up in your php.ini file. This is much faster than storing sessions in databases, and allows your sessions to persist across multiple web hosts (if you're in a load balanced situation)... this will also give your site a bit of a performance boost as there's no need to hit the filesystem / database for session info on every request.
- 您甚至可以将用户会话存储在memcache中 - 只需将其设置在php.ini文件中即可。这比在数据库中存储会话要快得多,并且允许您的会话在多个Web主机上保持不变(如果您处于负载平衡状态)...这也将为您的站点提供一点性能提升,因为不需要在每个请求上点击文件系统/数据库以获取会话信息。
... and many more ;)
... 还有很多 ;)
As to some of your concerns about memory footprint of individual cached items you've got a few options. My initial thought is to just give it a whirl, see how big these cache items really get (you can find several open-source things to monitor the actual cache usage, such as cacti). I think they'll be smaller than you'd think.
至于您对单个缓存项目的内存占用的一些担忧,您有一些选择。我最初的想法是给它一个旋转,看看这些缓存项目真正有多大(你可以找到几个开源的东西来监控实际的缓存使用情况,比如cacti)。我认为它们会比你想象的要小。
If they're not, I'd suggest re-thinking your cache strategy as far as what you actually cache, for how long, etc. Maybe you could build the feed from several things already in the cache (i.e. cache individual user data, and then build the feed for a person from all those individual items in cache). There are a lot of good articles out there on that front, just search 'em out :)
如果他们不是,我建议你重新思考你的缓存策略,直到实际缓存,持续多长时间等等。也许你可以从缓存中的几个东西构建feed(即缓存单个用户数据,然后从缓存中的所有单个项目构建一个人的订阅源。在这方面有很多好文章,只是搜索出来:)
#2
1
The default maximum object size that is allowed in Memcache is 1MB.
Memcache中允许的默认最大对象大小为1MB。
@jsaondavis : "session data is saved into a file".
@jsaondavis:“会话数据保存到文件中”。
Your above statement is wrong. Session can be configured to store in database. Default session hadndler is file.
您的上述陈述是错误的。会话可以配置为存储在数据库中。默认会话hadndler是文件。
#3
1
Redis would be a good solution:
Redis将是一个很好的解决方案:
Here is a thread on the Redis Vs. Memcached. Sounds like Redis has 512mb storage instead of the 1mb limit... QUITE a bit different :)
这是Redis Vs上的一个主题。 Memcached的。听起来像Redis有512mb的存储而不是1mb的限制......有点不同:)
Memcached与Redis?