最好的测试方法是什么?

I'm going through the EdgeCase Ruby Koans. In about_dice_project.rb, there's a test called "test_dice_values_should_change_between_rolls", which is straightforward:

我要讲的是EdgeCase Ruby Koans。在about_dice_project。rb，有一个测试叫做“test_dice_values_should_change_en_rolls”，这很简单:

  def test_dice_values_should_change_between_rolls
    dice = DiceSet.new

    dice.roll(5)
    first_time = dice.values

    dice.roll(5)
    second_time = dice.values

    assert_not_equal first_time, second_time,
      "Two rolls should not be equal"
  end

Except for this comment that appears there:

除了这里出现的评论:

# THINK ABOUT IT:
#
# If the rolls are random, then it is possible (although not
# likely) that two consecutive rolls are equal.  What would be a
# better way to test this.

Which (obviously) got me thinking: what is the best way to reliably test something random like that (specifically, and generally)?

这(显然)让我想到了:可靠地测试一些随机的东西(特别是，一般地)的最佳方法是什么?

13 个解决方案

#1

I'd say the best way to test anything that involves randomness is statistically. Run your dice function in a loop a million times, tabulate the results, and then run some hypothesis tests on the results. A million samples should give you enough statistical power that almost any deviations from correct code will be noticed. You are looking to demonstrate two statistical properties:

我想说，检验任何涉及随机性的东西的最好方法是统计上的。在一个循环中运行你的骰子函数一百万次，将结果列表，然后对结果进行一些假设检验。100万个样本应该会给你足够的统计能力，几乎任何偏离正确代码的偏差都会被注意到。您希望演示两个统计特性:

1. The probability of each value is what you intended it to be.
2. 每个值的概率都是你想要的。
3. All rolls are mutually independent events.
4. 所有的滚动都是相互独立的事件。

You can test whether the frequencies of the dice rolls are approximately correct using Pearson's Chi-square test. If you're using a good random nunber generator, such as the Mersenne Twister (which is the default in the standard lib for most modern languages, though not for C and C++), and you're not using any saved state from previous rolls other than the Mersenne Twister generator itself, then your rolls are for all practical purposes independent of one another.

您可以使用Pearson的卡方检验来测试掷骰的频率是否接近正确。如果你使用一个好的随机nunber生成器,梅森素数捻线机等(这是默认在大多数现代编程语言的标准*,虽然不是C和c++),和你不使用任何保存的状态从先前卷以外的梅森素数捻线机发电机本身,那么你的卷实际上是相互独立的。

As another example of statistical testing of random functions, when I ported the NumPy random number generators to the D programming language, my test for whether the port was correct was to use the Kolmogorov-Smirnov test to see whether the numbers generated matched the probability distributions they were supposed to match.

统计测试的随机函数的另一个例子,当我移植NumPy随机数生成器的D编程语言,我的测试端口是否正确是使用Kolmogorov-Smirnov测试是否匹配概率分布生成的数字应该匹配。

#2

IMHO most answers so far have missed the point of the Koan question, with the exception of @Super_Dummy. Let me elaborate on my thinking...

到目前为止，除了@Super_Dummy之外，IMHO的大多数回答都忽略了Koan问题的要点。让我详细阐述一下我的想法……

Say that instead of dice, we were flipping coins. Add on another constraint of only using one coin in our set, and we have a minimum non-trivial set that can generate "random" results.

我们不是掷骰子，而是掷硬币。再加上另一个限制，即在我们的集合中只使用一个硬币，我们有一个最小的非平凡集合，可以生成“随机”的结果。

If we wanted to check that flipping the "coin set" [in this case a single coin] generated a different result each time, we would expect the values of each separate result to be the same 50% of the time, on a statistical basis. Running that unit test through n iterations for some large n will simply exercise the PRNG. It tells you nothing of substance about the actual equality or difference between the two results.

如果我们想要检查每次投掷“硬币集”(在这种情况下是一枚硬币)会产生不同的结果，我们就会期望在统计的基础上，每个独立结果的值有50%是相同的。通过n个大n的迭代运行单元测试只需要使用PRNG。它没有告诉你这两个结果之间的实际相等或不同的实质。

To put it another way, in this Koan we're not actually concerned with the values of each roll of the dice. We're really more concerned that the returned rolls are actually representations of different rolls. Checking that the returned values are different is only a first-order check.

换句话说，在这个Koan中我们实际上并不关心每一掷骰子的值。我们更关心的是，返回的卷实际上是不同卷的表示。检查返回值是否不同只是一阶检查。

Most of the time that will be sufficient - but very occasionally, randomness could cause your unit test to fail. That's not a Good Thing™.

大多数时候，这就足够了——但偶尔，随机性会导致单元测试失败。这不是一件好事™。

If, in the case that two consecutive rolls return identical results, we should then check that the two results are actually represented by different objects. This would allow us to refactor the code in future [if that was needed], while being confident that the tests would still always catch any code that didn't behave correctly.

如果，如果两个连续的滚动返回相同的结果，那么我们应该检查这两个结果实际上是由不同的对象表示的。这将允许我们在将来重构代码(如果需要的话)，同时确信测试仍然会捕获任何不正确的代码。

TL;DR?

TL,博士?

def test_dice_values_should_change_between_rolls
  dice = DiceSet.new

  dice.roll(5)
  first_time = dice.values

  dice.roll(5)
  second_time = dice.values

  assert_not_equal [first_time, first_time.object_id],
    [second_time, second_time.object_id], "Two rolls should not be equal"

  # THINK ABOUT IT:
  #
  # If the rolls are random, then it is possible (although not
  # likely) that two consecutive rolls are equal.  What would be a
  # better way to test this.
end

#3

There is no way to write a state-based test for randomness. They are contradictory, since state-based tests proceed by giving known inputs and checking output. If your input (random seed) is unknown, there is no way to test.

没有办法为随机性编写基于状态的测试。它们是相互矛盾的，因为基于状态的测试通过给出已知的输入和检查输出来进行。如果您的输入(随机种子)是未知的，则没有办法进行测试。

Luckily, you don't really want to test the implementation of rand for Ruby, so you can just stub it out with an expectation using mocha.

幸运的是，您并不是真的想要测试Ruby的rand实现，所以您可以使用mocha将它用期望存根化。

def test_roll
  Kernel.expects(:rand).with(5).returns(1)
  Diceset.new.roll(5)
end

#4

It seems like there are 2 separate units here. First, a random number generator. Second, a "dice" abstraction that uses the (P)RNG.

这里好像有两个独立的单位。首先，一个随机数生成器。第二，使用(P)RNG的“骰子”抽象。

If you want to unit test the dice abstraction, then mock out the PRNG calls, and make sure it calls them, and returns an appropriate value for the input you give, etc.

如果您想要对骰子的抽象进行单元测试，那么可以模拟PRNG调用，并确保它调用它们，并为您提供的输入返回一个适当的值，等等。

The PRNG is probably part of your library/framework/OS and so I wouldn't bother testing it. Maybe you'll want an integration test to see if it returns reasonable values, but that's a whole 'nother problem.

PRNG可能是您的库/框架/OS的一部分，所以我不需要测试它。也许你想要一个集成测试，看看它是否返回合理的值，但这是一个完全没有问题的问题。

#5

Instead of comparing values, compare object_id:

不是比较值，而是比较object_id:

    assert_not_equal first_time.object_id, second_time.object_id

This assumes that other tests will check for array of integers.

这假定其他测试将检查整数数组。

#6

My solution was to allow a block to be passed to the roll function.

我的解决方案是允许一个块被传递给滚动函数。

class DiceSet
  def roll(n)
    @values = (1..n).map { block_given? ? yield : rand(6) + 1 }
  end
end

I can then pass my own RNG into the tests like this.

然后，我可以通过自己的RNG来进行这样的测试。

dice = DiceSet.net
dice.roll(5) { 1 }
first_result = dice.values
dice.roll(5) { 2 }
second_result = dice.values
assert_not_equal first_result, second_result

I don't know if that's really better, but it does abstract out the calls to the RNG. And it doesn't change the standard functionality.

我不知道这是否真的更好，但它确实抽象出了对RNG的调用。它不会改变标准功能。

#7

Just create new array each time roll method called. This way you can use

每次调用滚动方法时都要创建一个新的数组。你可以用这种方法

assert_not_same first_time, second_time,
"Two rolls should not be equal"

to test object_id equality. Yes, this test depends on implementation, but there is no way to test randomness. Other approach is to use mocks as floyd suggested.

测试object_id平等。是的，这个测试依赖于实现，但是没有办法测试随机性。另一种方法是使用floyd建议的mock。

#8

It seems a bit silly, to me. Are you supposed to be testing that the (psuedo) random number generator is generating random numbers? That's futile and pointless. If anything, you could test that dice.roll calls to your PRNG.

在我看来，这有点傻。您是否应该测试(psuedo)随机数生成器生成随机数?这是徒劳的和毫无意义的。如果有的话，你可以测试那个骰子。点名。

#9

I solved the problem using recursion:

我用递归解决了这个问题:

def roll times, prev_roll=[]
    @values.clear
    1.upto times do |n|
       @values << rand(6) + 1
    end
    roll(times, prev_roll) if @values == prev_roll
end

And had to add a dup method to the test variable, so it doesn't pass the reference to my instance variable @values.

并且必须向测试变量添加一个dup方法，这样它就不会传递对实例变量@values的引用。

def test_dice_values_should_change_between_rolls
    dice = DiceSet.new

    dice.roll(5)
    first_time = dice.values.dup

    dice.roll(5, first_time)
    second_time = dice.values

    assert_not_equal first_time, second_time,
       "Two rolls should not be equal"

  end

#10

rand is deterministic and depends on its seed. Use srand with a given number before the first roll and srand with a different number before the second roll. That would prevent repeating the series.

兰德是决定性的，取决于它的种子。在第一卷前使用给定的数字作为srand，在第二卷前使用不同的数字作为srand。这将避免重复这个系列。

srand(1)
dice.roll(5)
first_time = dice.values

srand(2)
dice.roll(5)
second_time = dice.values

assert_not_equal first_time, second_time,
  "Two rolls should not be equal"

#11

IMHO, randomness should be tested with dependency injection.

IMHO，随机性应该通过依赖注入进行测试。

Jon Skeet answered to the general answer of how to test randomness here

Jon Skeet回答了如何测试随机性的一般答案

I suggest you treat your source of randomness (a random number generator or whatever) as a dependency. Then you can test it with known inputs by providing either a fake RNG or one with a known seed. That removes the randomness from the test, while keeping it in the real code.

我建议您将随机性的来源(随机数生成器或其他)视为一个依赖项。然后，您可以通过提供假的RNG或带有已知种子的RNG来测试它。这将从测试中删除随机性，同时将其保留在实际代码中。

Example code of in our case may look something like this:

本例中的示例代码可能如下所示:

class DependentDiceSet
  attr_accessor :values, :randomObject

  def initialize(randomObject)
    @randomObject = randomObject
  end

  def roll(count)
    @values = Array.new(count) { @randomObject.userRand(1...6) }
  end
end

class MyRandom
  def userRand(values)
    return 6
  end
end

class RubyRandom
  def userRand(values)
    rand(values)
  end
end

A user can inject any random behavior and test that the dice are rolled by that behavior. I implement ruby random behavior and another one that return always 6.

用户可以注入任何随机行为并测试该行为是否滚动骰子。我实现了ruby的随机行为和另一个总是返回6的行为。

Usage:

用法:

randomDice = DependentDiceSet.new(RubyRandom.new)
sixDice = DependentDiceSet.new(MyRandom.new)

#12

i just created a new instance

我刚刚创建了一个新实例

def test_dice_values_should_change_between_rolls
    dice1 = DiceSet.new
    dice2 = DiceSet.new

    dice1.roll(5)
    first_time = dice1.values.dup

    dice2.roll(5, first_time)
    second_time = dice2.values

assert_not_equal first_time, second_time,
   "Two rolls should not be equal"

  end

#13

I solved it by simply creating a new set of values for each dice anytime the 'roll' method is called:

我通过简单地为每个骰子创建一组新的值来解决这个问题，只要“滚动”方法被调用:

def roll(n)     
    @numbers = []
    n.times do
      @numbers << rand(6)+1
    end
end

---