I have a rather small (ca. 4.5k pageviews a day) website running on Django, with PostgreSQL 8.3 as the db.
我有一个相当小的(每天大约4.5k的综合浏览量)网站在Django上运行,PostgreSQL 8.3作为数据库。
I am using the database as both the cache and the sesssion backend. I've heard a lot of good things about using Memcached for this purpose, and I would definitely like to give it a try. However, I would like to know exactly what would be the benefits of such a change: I imagine that my site may be just not big enough for the better cache backend to make a difference. The point is: it wouldn't be me who would be installing and configuring memcached, and I don't want to waste somebody's time for nothing or very little.
我使用数据库作为缓存和sesssion后端。为此我已经听说过很多关于使用Memcached的好东西,我肯定想尝试一下。但是,我想确切地知道这种改变会带来什么好处:我认为我的网站可能不够大,无法让更好的缓存后端有所作为。关键是:我不会安装和配置memcached,我不想浪费任何时间或非常少的时间。
How can I measure the overhead introduced by using the db as the cache backend? I've looked at django-debug-toolbar, but if I understand correctly it isn't something you'd like to put on a production site (you have to set DEBUG=True for it to work). Unfortunately, I cannot quite reproduce the production setting on my laptop (I have a different OS, CPU and a lot more RAM).
如何衡量使用db作为缓存后端引入的开销?我看过django-debug-toolbar,但如果我理解正确,那么你不想把它放在生产网站上(你必须设置DEBUG = True才能正常工作)。不幸的是,我不能完全重现我的笔记本电脑上的生产设置(我有不同的操作系统,CPU和更多的内存)。
Has anyone benchmarked different Django cache/session backends? Does anybody know what would be the performance difference if I was doing, for example, one session-write on every request?
有没有人对不同的Django缓存/会话后端进行基准测试?有没有人知道如果我正在做什么会有什么性能差异,例如,每次请求都有一次会话写入?
5 个解决方案
#1
At my previous work we tried to measure caching impact on site we was developing. On the same machine we load-tested the set of 10 pages that are most commonly used as start pages (object listings), plus some object detail pages taken randomly from the pool of ~200000. The difference was like 150 requests/second to 30000 requests/second and the database queries dropped to 1-2 per page.
在我之前的工作中,我们试图测量缓存对我们正在开发的网站的影响。在同一台机器上,我们负载测试了最常用作起始页(对象列表)的10页的集合,以及从~200000池中随机获取的一些对象详细页面。差异就像150请求/秒到30000请求/秒,数据库查询下降到每页1-2。
What was cached:
什么是缓存:
- sessions
- lists of objects retrieved for each individual page in object listing
- secondary objects and common content (found on each page)
- lists of object categories and other categorising properties
- object counters (calculated offline by cron job)
- individual objects
在对象列表中为每个单独页面检索的对象列表
次要对象和常见内容(在每页上找到)
对象类别列表和其他分类属性
对象计数器(由cron作业离线计算)
In general, we used only low-level granular caching, not the high-level cache framework. It required very careful design (cache had to be properly invalidated upon each database state change, like adding or modifying any object).
通常,我们只使用低级粒度缓存,而不是高级缓存框架。它需要非常仔细的设计(缓存必须在每次数据库状态更改时正确无效,如添加或修改任何对象)。
#2
The DiskCache project publishes Django cache benchmarks comparing local memory, Memcached, Redis, file based, and diskcache.DjangoCache. An added benefit of DiskCache is that no separate process is necessary (unlike Memcached and Redis). Instead cache keys and small values are memory-mapped into the Django process memory. Retrieving values from the cache is generally faster than Memcached on localhost. A number of settings control how much data is kept in memory; the rest being paged out to disk.
DiskCache项目发布Django缓存基准测试,比较本地内存,Memcached,Redis,基于文件和diskcache.DjangoCache。 DiskCache的另一个好处是不需要单独的进程(与Memcached和Redis不同)。相反,缓存键和小值被内存映射到Django进程内存中。从缓存中检索值通常比localhost上的Memcached快。许多设置控制内存中保存的数据量;其余的被分页到磁盘。
#3
Short answer : If you have enougth ram, memcached will be always faster. You can't really benchhmark memcached vs. database cache, just keep in mind that the big bottleneck with servers is disk access, specially write access.
简短的回答:如果你有使用ram,那么memcached总是会更快。你无法真正对memcached与数据库缓存进行基准测试,请记住,服务器的主要瓶颈是磁盘访问,特别是写访问。
Anyway, disk cache is better if you have many objects to cache and long time expiration. But for this situation, if you want gig performances, it is better to generate your pages statically with a python script and deliver them with ligthtpd or nginx.
无论如何,如果您有许多要缓存的对象并且很长时间到期,则磁盘缓存会更好。但是对于这种情况,如果你想要演出,最好用python脚本静态生成你的页面并用ligthtpd或nginx传递它们。
For memcached, you could adjust the amount of ram dedicated to the server.
对于memcached,您可以调整专用于服务器的ram数量。
#4
Just try it out. Use firebug or a similar tool and run memcache with a bit of RAM allocation (e.g. 64mb) on the test server.
试一试吧。使用firebug或类似工具,并在测试服务器上运行带有一点RAM分配(例如64mb)的内存缓存。
Mark your average loading results seen in firebug without memcache, then turn caching on and mark new results. That's done as easy as it said.
标记在没有memcache的情况下在firebug中看到的平均加载结果,然后打开缓存并标记新结果。这样做就像它说的一样简单。
The results usually shocks people, because the perfomance raises up very nicely.
结果通常会震撼人们,因为性能提升得非常好。
#5
Use django-debug-toolbar to see how much time has been saved on SQL query
使用django-debug-toolbar查看SQL查询保存了多少时间
#1
At my previous work we tried to measure caching impact on site we was developing. On the same machine we load-tested the set of 10 pages that are most commonly used as start pages (object listings), plus some object detail pages taken randomly from the pool of ~200000. The difference was like 150 requests/second to 30000 requests/second and the database queries dropped to 1-2 per page.
在我之前的工作中,我们试图测量缓存对我们正在开发的网站的影响。在同一台机器上,我们负载测试了最常用作起始页(对象列表)的10页的集合,以及从~200000池中随机获取的一些对象详细页面。差异就像150请求/秒到30000请求/秒,数据库查询下降到每页1-2。
What was cached:
什么是缓存:
- sessions
- lists of objects retrieved for each individual page in object listing
- secondary objects and common content (found on each page)
- lists of object categories and other categorising properties
- object counters (calculated offline by cron job)
- individual objects
在对象列表中为每个单独页面检索的对象列表
次要对象和常见内容(在每页上找到)
对象类别列表和其他分类属性
对象计数器(由cron作业离线计算)
In general, we used only low-level granular caching, not the high-level cache framework. It required very careful design (cache had to be properly invalidated upon each database state change, like adding or modifying any object).
通常,我们只使用低级粒度缓存,而不是高级缓存框架。它需要非常仔细的设计(缓存必须在每次数据库状态更改时正确无效,如添加或修改任何对象)。
#2
The DiskCache project publishes Django cache benchmarks comparing local memory, Memcached, Redis, file based, and diskcache.DjangoCache. An added benefit of DiskCache is that no separate process is necessary (unlike Memcached and Redis). Instead cache keys and small values are memory-mapped into the Django process memory. Retrieving values from the cache is generally faster than Memcached on localhost. A number of settings control how much data is kept in memory; the rest being paged out to disk.
DiskCache项目发布Django缓存基准测试,比较本地内存,Memcached,Redis,基于文件和diskcache.DjangoCache。 DiskCache的另一个好处是不需要单独的进程(与Memcached和Redis不同)。相反,缓存键和小值被内存映射到Django进程内存中。从缓存中检索值通常比localhost上的Memcached快。许多设置控制内存中保存的数据量;其余的被分页到磁盘。
#3
Short answer : If you have enougth ram, memcached will be always faster. You can't really benchhmark memcached vs. database cache, just keep in mind that the big bottleneck with servers is disk access, specially write access.
简短的回答:如果你有使用ram,那么memcached总是会更快。你无法真正对memcached与数据库缓存进行基准测试,请记住,服务器的主要瓶颈是磁盘访问,特别是写访问。
Anyway, disk cache is better if you have many objects to cache and long time expiration. But for this situation, if you want gig performances, it is better to generate your pages statically with a python script and deliver them with ligthtpd or nginx.
无论如何,如果您有许多要缓存的对象并且很长时间到期,则磁盘缓存会更好。但是对于这种情况,如果你想要演出,最好用python脚本静态生成你的页面并用ligthtpd或nginx传递它们。
For memcached, you could adjust the amount of ram dedicated to the server.
对于memcached,您可以调整专用于服务器的ram数量。
#4
Just try it out. Use firebug or a similar tool and run memcache with a bit of RAM allocation (e.g. 64mb) on the test server.
试一试吧。使用firebug或类似工具,并在测试服务器上运行带有一点RAM分配(例如64mb)的内存缓存。
Mark your average loading results seen in firebug without memcache, then turn caching on and mark new results. That's done as easy as it said.
标记在没有memcache的情况下在firebug中看到的平均加载结果,然后打开缓存并标记新结果。这样做就像它说的一样简单。
The results usually shocks people, because the perfomance raises up very nicely.
结果通常会震撼人们,因为性能提升得非常好。
#5
Use django-debug-toolbar to see how much time has been saved on SQL query
使用django-debug-toolbar查看SQL查询保存了多少时间