如何禁用Django查询缓存?

时间:2022-03-23 02:28:05

In my Django application, I repeatedly run the same query on my database (e.g. every 10 seconds). I then create an MD5 sum over the queryset I receive and compare that to the MD5 sum I created in the previous run. If both are equal, the data has not changed and the web page does not need updating.

在Django应用程序中,我反复在数据库上运行相同的查询(例如,每10秒)。然后,我在收到的queryset上创建一个MD5求和,并将其与前一次运行中创建的MD5求和进行比较。如果两者相等,则数据没有更改,web页面不需要更新。

While I do this, the data in the DB might change.

当我这样做时,DB中的数据可能会改变。

However, the query returns the same queryset, apparently due to query caching.

但是,查询返回相同的queryset,显然是由于查询缓存。

How can I disable the query cache and explicitely execute the query on the DB ?

如何禁用查询缓存并在DB上显式地执行查询?

5 个解决方案

#1


41  

I came across behavior that I thought was some kind of caching, but it turned out to be database transactions fooling me.

我遇到了一些行为,我认为是某种缓存,但结果是数据库事务欺骗了我。

I had the problem where in another process, items were get added to the database, and I wanted to monitor progress of the other process, so I opened up a django shell and issued the following:

我有一个问题,在另一个过程中,项目被添加到数据库中,我想要监视另一个进程的进度,所以我打开了一个django shell并发布了如下内容:

>>> MyData.objects.count()
74674

>>> MyData.objects.count()
74674

The value wasn't changing, even though it actually was in the database. I realized that at least with the way I had MySQL & django setup that I was in a transaction and would only see a "snapshot" of the database at the time I opened the transaction.

这个值没有变化,即使它实际上在数据库中。我意识到,至少通过使用MySQL和django的设置,我处于事务中,并且在打开事务时只能看到数据库的“快照”。

Since with views in django, I had autocommit behavior defined, this was fine for each view to only see a snapshot, as the next time a view was called it would be in a different transaction. But for a piece of code that was not automatically committing, it would not see any changes in the db except those that were made in this transaction.

因为对于django中的视图,我定义了自动提交行为,所以对于每个视图来说,这都可以只看到快照,因为下次调用视图时,它将位于不同的事务中。但是对于没有自动提交的代码片段,除了在该事务中所做的更改外,它不会看到db中的任何更改。

Just thought I would toss this answer in for anyone who may come upon this situation.

我只是想把这个答案抛给任何可能遇到这种情况的人。

To solve, commit your transaction, which can be manually done like so:

要解决这个问题,提交您的事务,您可以这样做:

>> from django.db import transaction
>> transaction.enter_transaction_management()
>> transaction.commit() # Whenever you want to see new data

#2


9  

Query caching only applies within a QuerySet. In other words, if you evaluate the same queryset object twice, query caching will operate. But if you are doing a query every 10 seconds, presumably this is via a cron that spawns a new process each time, so there is no way Django will cache anything.

查询缓存仅适用于查询集。换句话说,如果您对同一个queryset对象进行两次评估,那么查询缓存就会运行。但是,如果您每10秒执行一次查询,那么假设这是通过cron进行的,每次都会生成一个新进程,因此Django不可能缓存任何内容。

It is possible that your database's own cache will come into operation if you're repeatedly performing exactly the same query. You should look at the documentation for your DBMS to see how to manage that properly.

如果重复执行相同的查询,您的数据库的缓存可能会开始运行。您应该查看数据库管理系统的文档,以了解如何正确地管理它。

#3


6  

The link you provide to the Django Documentation implies that the following:

您提供给Django文档的链接意味着:

>>> print [e.headline for e in Entry.objects.all()]
>>> print [e.pub_date for e in Entry.objects.all()]

creates two queries to the database, whilst:

对数据库创建两个查询,同时:

>>> queryset = Poll.objects.all()
>>> print [p.headline for p in queryset] # Evaluate the query set.
>>> print [p.pub_date for p in queryset] # Re-use the cache from the evaluation.

uses the query cache, as you are accessing the same evaluation results.

使用查询缓存,因为您正在访问相同的计算结果。

#4


1  

Thank you very much for your answers, your replies made me take a few steps back and rethink.

非常感谢你的回答,你的回复让我后退了几步,重新思考。

In order to test caching on a DBMS level, I went away from Django and used a shell script I anyway had handy to periodically query data from a SQLite db, while I added data in a second shell session. The new data showed up in the periodic queries right after I added them, so no query caching here.

为了在DBMS级别上测试缓存,我离开了Django,使用了一个shell脚本,可以随时从SQLite db中查询数据,同时在第二个shell会话中添加数据。新数据在我添加后的定期查询中出现,所以这里没有查询缓存。

This narrowed it down to the Django part. The code involved is not very complex and a little log output and a code review revealed the problem: The query used to obtain the queryset used in turn to create the MD5 sum had an error and was always empty. Therefore, the MD5 sum was always the same. Did indeed look like a cached result - data is changing, but the queryset stays the same. The problem did not show in the application, as a different query was used to obtain data displayed there.

这缩小到Django部分。所涉及的代码并不复杂,少量的日志输出和代码检查揭示了问题:用于获取用于创建MD5和的queryset的查询有一个错误,并且始终为空。因此,MD5的和总是相同的。确实看起来像缓存的结果——数据正在改变,但是queryset保持不变。这个问题在应用程序中没有显示,因为使用了不同的查询来获取显示在那里的数据。

Lesson learned: If you're completely puzzled, take a step back and re-think your assumptions.

教训:如果你完全困惑,退一步,重新思考你的假设。

Thanks again! :-)

再次感谢!:-)

#5


0  

I met this problem on django version 1.8. There is not direct way to do it, but there are some ways to make the queryset re-evaluated and executed by accessing db instead of cache. I found it in Django Queryset Documentation

我在django版本1.8上遇到了这个问题。没有直接的方法,但是有一些方法可以通过访问db而不是缓存来重新计算和执行queryset。我在Django Queryset文档中找到了它

I used one of them to handle my problem. It is exists() function of querysets. len() and repr() can also be used. They worked for me too.

我用其中一个来解决我的问题。它是querysets的exist()函数。len()和repr()也可以使用。他们也为我工作。

Example

例子

queryset = ModelClass.objects.filter(....)
queryset.exists()

#or len(queryset)
#or repr(queryset)

#Now queryset is re-evaluated. 

#1


41  

I came across behavior that I thought was some kind of caching, but it turned out to be database transactions fooling me.

我遇到了一些行为,我认为是某种缓存,但结果是数据库事务欺骗了我。

I had the problem where in another process, items were get added to the database, and I wanted to monitor progress of the other process, so I opened up a django shell and issued the following:

我有一个问题,在另一个过程中,项目被添加到数据库中,我想要监视另一个进程的进度,所以我打开了一个django shell并发布了如下内容:

>>> MyData.objects.count()
74674

>>> MyData.objects.count()
74674

The value wasn't changing, even though it actually was in the database. I realized that at least with the way I had MySQL & django setup that I was in a transaction and would only see a "snapshot" of the database at the time I opened the transaction.

这个值没有变化,即使它实际上在数据库中。我意识到,至少通过使用MySQL和django的设置,我处于事务中,并且在打开事务时只能看到数据库的“快照”。

Since with views in django, I had autocommit behavior defined, this was fine for each view to only see a snapshot, as the next time a view was called it would be in a different transaction. But for a piece of code that was not automatically committing, it would not see any changes in the db except those that were made in this transaction.

因为对于django中的视图,我定义了自动提交行为,所以对于每个视图来说,这都可以只看到快照,因为下次调用视图时,它将位于不同的事务中。但是对于没有自动提交的代码片段,除了在该事务中所做的更改外,它不会看到db中的任何更改。

Just thought I would toss this answer in for anyone who may come upon this situation.

我只是想把这个答案抛给任何可能遇到这种情况的人。

To solve, commit your transaction, which can be manually done like so:

要解决这个问题,提交您的事务,您可以这样做:

>> from django.db import transaction
>> transaction.enter_transaction_management()
>> transaction.commit() # Whenever you want to see new data

#2


9  

Query caching only applies within a QuerySet. In other words, if you evaluate the same queryset object twice, query caching will operate. But if you are doing a query every 10 seconds, presumably this is via a cron that spawns a new process each time, so there is no way Django will cache anything.

查询缓存仅适用于查询集。换句话说,如果您对同一个queryset对象进行两次评估,那么查询缓存就会运行。但是,如果您每10秒执行一次查询,那么假设这是通过cron进行的,每次都会生成一个新进程,因此Django不可能缓存任何内容。

It is possible that your database's own cache will come into operation if you're repeatedly performing exactly the same query. You should look at the documentation for your DBMS to see how to manage that properly.

如果重复执行相同的查询,您的数据库的缓存可能会开始运行。您应该查看数据库管理系统的文档,以了解如何正确地管理它。

#3


6  

The link you provide to the Django Documentation implies that the following:

您提供给Django文档的链接意味着:

>>> print [e.headline for e in Entry.objects.all()]
>>> print [e.pub_date for e in Entry.objects.all()]

creates two queries to the database, whilst:

对数据库创建两个查询,同时:

>>> queryset = Poll.objects.all()
>>> print [p.headline for p in queryset] # Evaluate the query set.
>>> print [p.pub_date for p in queryset] # Re-use the cache from the evaluation.

uses the query cache, as you are accessing the same evaluation results.

使用查询缓存,因为您正在访问相同的计算结果。

#4


1  

Thank you very much for your answers, your replies made me take a few steps back and rethink.

非常感谢你的回答,你的回复让我后退了几步,重新思考。

In order to test caching on a DBMS level, I went away from Django and used a shell script I anyway had handy to periodically query data from a SQLite db, while I added data in a second shell session. The new data showed up in the periodic queries right after I added them, so no query caching here.

为了在DBMS级别上测试缓存,我离开了Django,使用了一个shell脚本,可以随时从SQLite db中查询数据,同时在第二个shell会话中添加数据。新数据在我添加后的定期查询中出现,所以这里没有查询缓存。

This narrowed it down to the Django part. The code involved is not very complex and a little log output and a code review revealed the problem: The query used to obtain the queryset used in turn to create the MD5 sum had an error and was always empty. Therefore, the MD5 sum was always the same. Did indeed look like a cached result - data is changing, but the queryset stays the same. The problem did not show in the application, as a different query was used to obtain data displayed there.

这缩小到Django部分。所涉及的代码并不复杂,少量的日志输出和代码检查揭示了问题:用于获取用于创建MD5和的queryset的查询有一个错误,并且始终为空。因此,MD5的和总是相同的。确实看起来像缓存的结果——数据正在改变,但是queryset保持不变。这个问题在应用程序中没有显示,因为使用了不同的查询来获取显示在那里的数据。

Lesson learned: If you're completely puzzled, take a step back and re-think your assumptions.

教训:如果你完全困惑,退一步,重新思考你的假设。

Thanks again! :-)

再次感谢!:-)

#5


0  

I met this problem on django version 1.8. There is not direct way to do it, but there are some ways to make the queryset re-evaluated and executed by accessing db instead of cache. I found it in Django Queryset Documentation

我在django版本1.8上遇到了这个问题。没有直接的方法,但是有一些方法可以通过访问db而不是缓存来重新计算和执行queryset。我在Django Queryset文档中找到了它

I used one of them to handle my problem. It is exists() function of querysets. len() and repr() can also be used. They worked for me too.

我用其中一个来解决我的问题。它是querysets的exist()函数。len()和repr()也可以使用。他们也为我工作。

Example

例子

queryset = ModelClass.objects.filter(....)
queryset.exists()

#or len(queryset)
#or repr(queryset)

#Now queryset is re-evaluated.