如何发现动态生成的测试用例？

This is a follow-up to a previous question of mine.

这是我前一个问题的后续行动。

In the previous question, methods were explored to implement what was essentially the same test over an entire family of functions, ensuring testing did not stop at the first function that failed.

在上一个问题中,探索了一些方法来实现对整个函数族基本上相同的测试,确保测试不会在第一个失败的函数处停止。

My preferred solution used a metaclass to dynamically insert the tests into a unittest.TestCase. Unfortunately, nose does not pick this up because nose statically scans for test cases.

我的首选解决方案使用元类将测试动态插入unittest.TestCase。不幸的是,鼻子不会选择这个,因为鼻子静态扫描测​​试用例。

How do I get nose to discover and run such a TestCase? Please refer here for an example of the TestCase in question.

如何发现并运行这样的TestCase?有关TestCase的示例,请参阅此处。

3 个解决方案

#1

Nose has a "test generator" feature for stuff like this. You write a generator function that yields each "test case" function you want it to run, along with its args. Following your previous example, this could check each of the functions in a separate test:

对于像这样的东西,Nose有一个“测试生成器”功能。您编写了一个生成器函数,该函数生成您希望它运行的每个“测试用例”函数及其args。按照上一个示例,这可以检查单独测试中的每个函数:

import unittest
import numpy

from somewhere import the_functions

def test_matrix_functions():
    for function in the_functions:
        yield check_matrix_function, function

def check_matrix_function(function)
    matrix1 = numpy.ones((5,10))
    matrix2 = numpy.identity(5)
    output = function(matrix1, matrix2)
    assert matrix1.shape == output.shape, \
           "%s produces output of the wrong shape" % str(function)

#2

Nose does not scan for tests statically, so you can use metaclass magic to make tests that Nose finds.

Nose不会静态扫描测​​试,因此您可以使用元类魔法进行Nose发现的测试。

The hard part is that standard metaclass techniques don't set the func_name attribute correctly, which is what Nose looks for when checking whether methods on your class are tests.

困难的部分是标准元类技术没有正确设置func_name属性,这是Nose在检查类中的方法是否为测试时查找的内容。

Here's a simple metaclass. It looks through the func dict and adds a new method for every method it finds, asserting that the method it found has a docstring. These new synthetic methods are given the names "test_%d" %i.

这是一个简单的元类。它查看了func dict并为它找到的每个方法添加了一个新方法,断言它找到的方法有一个docstring。这些新的合成方法的名称为“test_%d”%i。

import new
from inspect import isfunction, getdoc

class Meta(type):
    def __new__(cls, name, bases, dct):

        newdct = dct.copy()
        for i, (k, v) in enumerate(filter(lambda e: isfunction(e[1]), dct.items())):
            def m(self, func):
                assert getdoc(func) is not None

            fname = 'test_%d' % i
            newdct[fname] = new.function(m.func_code, globals(), fname,
                (v,), m.func_closure)

        return super(Meta, cls).__new__(cls, 'Test_'+name, bases, newdct)

Now, let's create a new class that uses this metaclass

现在,让我们创建一个使用此元类的新类

class Foo(object):
    __metaclass__ = Meta

    def greeter(self):
        "sdf"
        print 'Hello World'

    def greeter_no_docstring(self):
        pass

At runtime, Foo will actually be named Test_Foo and will have greeter, greeter_no_docstring, test_1 and test_2 as its methods. When I run nosetests on this file, here's the output:

在运行时,Foo实际上将命名为Test_Foo,并将以greeter,greeter_no_docstring,test_1和test_2作为其方法。当我在这个文件上运行nosetests时,这是输出:

$ nosetests -v test.py
test.Test_Foo.test_0 ... FAIL
test.Test_Foo.test_1 ... ok

======================================================================
FAIL: test.Test_Foo.test_0
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/rmcgibbo/Desktop/test.py", line 10, in m
    assert getdoc(func) is not None
AssertionError

----------------------------------------------------------------------
Ran 2 tests in 0.002s

FAILED (failures=1)

This metaclass isn't really useful as is, but if you instead use the Meta not as a proper metaclass, but as more of a functional metaclass (i.e. takes a class as an argument and returns a new class, one that's renamed so that nose will find it), then it is useful. I've used this approach to automatically test that the docstrings adhere to the Numpy standard as part of a nose test suite.

这个元类本身并不是很有用,但是如果你改为使用Meta作为一个合适的元类,而是作为一个更具功能性的元类(即将一个类作为一个参数并返回一个新类,一个重命名为一个会发现它,然后它是有用的。我已经使用这种方法自动测试docstrings是否符合Numpy标准作为鼻子测试套件的一部分。

Also, I've had a lot of trouble getting proper closure working with new.function, which is why this code uses m(self, func) where func is made to be a default argument. It would be more natural to use a closure over value, but that doesn't seem to work.

另外,我在使用new.function进行正确的闭包时遇到了很多麻烦,这就是为什么这段代码使用m(self,func),其中func是一个默认参数。使用闭包超过值更自然,但这似乎不起作用。

#3

You could try to generate the testcase classes with type()

您可以尝试使用type()生成测试用例类

class UnderTest_MixIn(object):

    def f1(self, i):
        return i + 1

    def f2(self, i):
        return i + 2

SomeDynamicTestcase = type(
    "SomeDynamicTestcase", 
    (UnderTest_MixIn, unittest.TestCase), 
    {"even_more_dynamic":"attributes .."}
)

# or even:

name = 'SomeDynamicTestcase'
globals()[name] = type(
    name, 
    (UnderTest_MixIn, unittest.TestCase), 
    {"even_more_dynamic":"attributes .."}
)

This should be created when nose tries to import your test_module so it should work.

这应该在nose尝试导入test_module时创建,以便它可以工作。

The advantage of this approach is that you can create many combinations of tests dynamically.

这种方法的优点是您可以动态创建许多测试组合。