Google AppEngine：日期范围未返回正确的结果

Im trying to search for some values within a date range for a specific type, but content for dates that exist in the database are not being returned by the query.

我试图在特定类型的日期范围内搜索某些值,但查询不会返回数据库中存在的日期的内容。

Here is an extract of the python code:

这是python代码的摘录:

deltaDays = timedelta(days= 20)
endDate = datetime.date.today()
startDate = endDate - deltaDays

result = db.GqlQuery(
   "SELECT * FROM myData WHERE mytype = :1 AND pubdate >= :2 and pubdate <= :3", type, startDate, endDate
  )

class myData(db.Model):
   mytype = db.StringProperty(required=True)
   value =  db.FloatProperty(required=True)
   pubdate = db.DateTimeProperty(required=True)

The GQL returns data, but some rows that I am expecting are missing:

GQL返回数据,但是我期待的某些行丢失了:

 2009-03-18 00:00:00
(missing date in results: 2009-03-20 data exists in database)
 2009-03-23 00:00:00
 2009-03-24 00:00:00
 2009-03-25 00:00:00
 2009-03-26 00:00:00
(missing date in results: 2009-03-27 data exists in database)
 2009-03-30 00:00:00
(missing date in results: 2009-03-31. data exists in database)
 2009-04-01 00:00:00
 2009-04-02 00:00:00
 2009-04-03 00:00:00
 2009-04-06 00:00:00

I uploaded the data via de bulkload script. I just can think of the indexes being corrupted or something similar. This same query used to work for another table i had. But i had to replace it with new content from another source, and this new content is not responding to the query in the same way. The table has around 700.000 rows if that makes any difference.

我通过de bulkload脚本上传了数据。我只能想到索引被破坏或类似的东西。这个查询曾经用于我的另一个表。但是我不得不用来自其他来源的新内容替换它,并且这个新内容没有以相同的方式响应查询。如果这有任何不同,该表有大约700.000行。

I have done more research ant it appears that its a bug in the appEngine DataStore. For more information about the bug check this link: http://code.google.com/p/googleappengine/issues/detail?id=901

我做了更多的研究,似乎它是appEngine DataStore中的一个错误。有关错误的详细信息,请查看以下链接:http://code.google.com/p/googleappengine/issues/detail?id = 901

I have tried droping the index and recreating it with no luck.

我试过删除索引并重新创建它没有运气。

thanks

1 个解决方案

#1

nothing looks wrong to me. are you sure that the missing dates also have mytype == type?

我没什么不好看的。你确定缺少的日期也有mytype == type吗?

i have observed some funny behaviour with indexes in the past. I recommend writing a handler to iterate through all of your records and just put() them back in the database. maybe something with the bulk uploader isn't working properly.

我在过去观察过一些有趣的行为。我建议编写一个处理程序来遍历所有记录,然后将它们放回数据库中。也许批量上传器的东西不能正常工作。

Here's the type of handler I use to iterate through all the entities in a model class:

这是我用来迭代模型类中所有实体的处理程序类型:

 class PPIterator(BaseRequestHandler):
  def get(self):
    query = Model.gql('ORDER BY __key__')
    last_key_str = self.request.get('last')
    if last_key_str:
      last_key = db.Key(last_key_str)
      query = Model.gql('WHERE __key__ > :1 ORDER BY __key__', last_key)
    entities = query.fetch(11)
    new_last_key_str = None
    if len(entities) == 11:
      new_last_key_str = str(entities[9].key())
    for e in entities:
      e.put()
    if new_last_key_str:
      self.response.out.write(json.write(new_last_key_str))
    else:
      self.response.out.write(json.write('done'))

You can use whatever you want to iterate through the entities. I used to use Javascript in a browser window, but found that was a pig when making hundreds of thousands of requests. These days I find it more convenient to use a ruby script like this one:

您可以使用任何想要迭代实体的内容。我曾经在浏览器窗口中使用Javascript,但发现在制作成千上万的请求时这是一头猪。这些天我发现使用像这样的ruby脚本更方便:

require 'net/http'
require 'json'
last=nil
while last != 'done'
  url = 'your_url'
  path = '/your_path'
  path += "?/last=#{last}" if last
  last = Net::HTTP.get(url,path)
  puts last
end

Ben

UPDATE: now that remote api is working and reliable, I rarely write this type of handler anymore. The same ideas apply to the code you'd use there to iterate through the entities in the remote api console.

更新:现在远程api工作可靠,我很少再写这种类型的处理程序了。相同的想法适用于您在那里使用的代码来迭代远程api控制台中的实体。

#1