为什么django测试只有在运行完整的测试套件时才会失败?

I have a test in Django 1.5 that passes in these conditions:

我在Django 1.5中有一个测试通过了这些条件:

when run by itself in isolation
当独立运行时
when the full TestCase is run
在运行完整的TestCase时。
when all of my app's tests are run
当我的应用程序的所有测试运行时

But it fails when the full test suite is run with python manage.py test. Why might this be happening?

但是当使用python管理运行完整的测试套件时，它会失败。py测试。为什么会这样?

The aberrant test uses django.test.Client to POST some data to an endpoint, and then the test checks that an object was successfully updated. Could some other app be modifying the test client or the data itself?

异常测试使用django.test。客户机将一些数据发布到端点，然后测试检查对象是否已成功更新。其他应用程序会修改测试客户端或数据本身吗?

I have tried some print debugging and I see all of the data being sent and received as expected. The specific failure is a does-not-exist exception that is raised when I try to fetch the to-be-updated object from the db. Strangely, in the exception handler itself, I can query for all objects of that type and see that the target object does in fact exist.

我已经尝试了一些打印调试，我看到所有的数据都按照预期发送和接收。特定的失败是一个不存在的异常，当我试图从数据库中获取要更新的对象时，它就会被抛出。奇怪的是，在异常处理程序本身中，我可以查询该类型的所有对象，并看到目标对象确实存在。

Edit:

编辑:

My issue was resolved when I found that I was querying for the target object by id and User and not id and UserProfile, but it's still confusing to me that this would work in some cases but fail in others.

当我发现我是通过id和User而不是id和UserProfile来查询目标对象时，我的问题得到了解决，但我仍然感到困惑的是，在某些情况下，这行得通，但在其他情况下行不通。

I also found that the test would fail with python manage.py test auth <myapp>

我还发现在python管理下测试会失败。py测试认证< myapp >

4 个解决方案

#1

It sounds like your problem does not involve mocks, but I just spent all day debugging an issue with similar symptoms, and your question is the first one that came up when I was searching for a solution, so I wanted to share my solution here, just in case it will prove helpful for others. In my case, the issue was as follows.

这听起来像你的问题不涉及嘲笑,但我只是花了一整天调试类似症状的问题,和你的问题是第一个出现在我寻找一个解决方案,所以我想在这里分享我的解决方案,以防它将会是很有帮助的。就我而言，问题是这样的。

I had a single test that would pass in isolation, but fail when run as part of my full test suite. In one of my view functions I was using the Django send_mail() function. In my test, rather than having it send me an email every time I ran my tests, I patched send_mail in my test method:

我有一个独立通过的测试，但是在作为整个测试套件的一部分运行时失败了。在我的一个视图函数中，我使用了Django send_mail()函数。在我的测试中，我没有在每次运行测试时都给我发邮件，而是在我的测试方法中修补了send_mail:

from mock import patch
...

def test_stuff(self):
    ...

    with patch('django.core.mail.send_mail') as mocked_send_mail:

    ...

That way, after my view function is called, I can test that send_mail was called with:

这样，在我的视图函数被调用之后，我可以测试send_mail的调用:

self.assertTrue(mocked_send_mail.called)

This worked fine when running the test on its own, but failed when run with other tests in the suite. The reason this fails is that when it runs as part of the suite other views are called beforehand, causing the views.py file to be loaded, causing send_mail to be imported before I get the chance to patch it. So when send_mail gets called in my view, it is the actual send_mail that gets called, not my patched version. When I run the test alone, the function gets mocked before it is imported, so the patched version ends up getting imported when views.py is loaded. This situation is described in the mock documentation, which I had read a few times before, but now understand quite well after learning the hard way...

这在单独运行测试时运行良好，但在与套件中的其他测试一起运行时失败。失败的原因是当它作为套件的一部分运行时，会预先调用其他视图，从而导致视图。py文件将被加载，导致在我获得补丁之前导入send_mail。因此，当在我的视图中调用send_mail时，调用的是实际的send_mail，而不是我的补丁版本。当我单独运行测试时，函数会在导入之前被模拟，所以当视图出现时，补丁版本会被导入。py加载。这种情况是在模拟文档中描述的，我以前读过几次，但是在学习了困难的方法之后，现在很好理解了……

The solution was simple: instead of patching django.core.mail.send_mail I just patched the version that had already been imported in my views.py - myapp.views.send_mail. In other words:

解决方法很简单:不去修补django.core.mail。send_mail我刚刚修补了视图中已经导入的版本。py - myapp.views.send_mail。换句话说:

with patch('myapp.views.send_mail') as mocked_send_mail:
...

#2

Try this to help you debug:

尝试以下方法来帮助您调试:

./manage.py test --reverse

In my case I realised that one test was updating certain data which would cause the following test to fail.

在我的例子中，我意识到一个测试正在更新某些数据，这将导致下面的测试失败。

#3

Another possibility is that you've disconnected signals in the setUp of a test class and did not re-connect in the tearDown. This explained my issue.

另一种可能是，您在测试类的设置中断开了信号，并且在tearDown中没有重新连接。这解释了我的问题。

#4

There is a lot of nondeterminism that can come from tests that involve the database.

有很多不确定性来自涉及数据库的测试。

For instance, most databases don't offer deterministic selects unless you do an order by. This leads to the strange behavior where when the stars align, the database returns things in a different order than you might expect, and tests that look like

例如，大多数数据库不提供确定性的选择，除非您执行一个order by。这导致了一种奇怪的行为:当星星对齐时，数据库以不同于预期的顺序返回数据，并进行类似的测试

result = pull_stuff_from_database()
assert result[0] == 1
assert result[1] == 2

will fail because result[0] == 2 and result[1] == 1.

将失败，因为结果[0]= 2，结果[1]= 1。

Another source of strange nondeterministic behavior is the id autoincrement together with sorting of some kind.

奇怪的不确定性行为的另一个来源是id自增以及某种排序。

Let's say each tests creates two items and you sort by item name before you do assertions. When you run it by itself, "Item 1" and "Item 2" work fine and pass the test. However, when you run the entire suite, one of the tests generates "Item 9" and "Item 10". "Item 10" is sorted ahead of "Item 9" so your test fails because the order is flipped.

假设每个测试创建两个项目，在进行断言之前，您要按项目名称进行排序。当你自己运行它，“项目1”和“项目2”工作良好，通过测试。但是，当您运行整个套件时，其中一个测试将生成“项目9”和“项目10”。“项目10”在“项目9”前面进行排序，因此您的测试失败，因为订单被翻转。

#1

from mock import patch
...

def test_stuff(self):
    ...

    with patch('django.core.mail.send_mail') as mocked_send_mail:

    ...