iOS测试/规格TDD/BDD以及集成与验收测试

What are the best technologies to use for behavior-driven development on the iPhone? And what are some open source example projects that demonstrate sound use of these technologies? Here are some options I've found:

在iPhone上使用行为驱动开发的最佳技术是什么?有哪些开放源码示例项目展示了这些技术的合理使用?以下是我发现的一些选项:

---

Unit Testing

Test::Unit Style

1. OCUnit/SenTestingKit as explained in iOS Development Guide: Unit Testing Applications & other OCUnit references.
   • Examples: iPhoneUnitTests, Three20
   • 例子:iPhoneUnitTests Three20
2. 在iOS开发指南中解释的OCUnit/SenTestingKit:单元测试应用程序和其他OCUnit引用。例子:iPhoneUnitTests Three20
3. CATCH
4. 抓
5. GHUnit
6. GHUnit
7. Google Toolbox for Mac: iPhone Unit Testing
8. Mac的谷歌工具箱:iPhone单元测试

RSpec Style

1. Kiwi (which also comes with mocking & expectations)
2. 猕猴桃(也带有嘲笑和期待)
3. Cedar
4. 雪松
5. Jasmine with UI Automation as shown in dexterous' iOS-Acceptance-Testing specs
6. 如灵巧的ios -可接受测试规格所示，具有UI自动化的Jasmine

---

Acceptance Testing

Selenium Style

1. UI Automation (works on device)

   UI自动化(在设备上工作)

   • UI Automation Instruments Guide
   • UI自动化工具指南
   • UI Automation reference documentation
   • UI自动化参考文档
   • Tuneup js - cool library for using with UIAutomation.
   • Tuneup js - cool库用于使用UIAutomation。

   • Capturing User Interface Actions into Automation Scripts

     将用户界面动作捕获到自动化脚本中

     It's possible to use Cucumber (written in JavaScript) to drive UI Automation. This would be a great open-source project. Then, we could write Gherkin to run UI Automation testing. For now, I'll just write Gherkin as comments.

     可以使用Cucumber(用JavaScript编写)来驱动UI自动化。这将是一个伟大的开源项目。然后，我们可以编写Gherkin来运行UI自动化测试。现在，我只写小黄瓜作为评论。

   UPDATE: Zucchini Framework seems to blend Cucumber & UI Automation! :)

   更新:Zucchini框架似乎融合了黄瓜和UI自动化!:)

   Old Blog Posts:

   古老的博文:

   • Alex Vollmer's UI Automation tutorial
   • Alex Vollmer的UI自动化教程。
   • O'Reilly Answers UI Automation tutorial
   • O'Reilly回答UI自动化教程。
   • Adi Saxena's UI Automation tutorial
   • Adi Saxena的UI自动化教程

2. UISpec with UISpecRunner

   与UISpecRunner UISpec

   • UISpec is open source on Google Code.
   • UISpec是谷歌代码上的开源。
   • UISpec has comprehensive documentation.
   • UISpec全面的文档。

3. FoneMonkey

   FoneMonkey

Cucumber Style

1. Frank and iCuke (based on the Cucumber meets iPhone talk)

   Frank和iCuke(基于Cucumber meets iPhone talk)

   • The Frank Google Group has much more activity than the iCuke Google Group.
   • Frank谷歌组比iCuke谷歌组活跃得多。
   • Frank runs on both device and simulator, while iCuke only runs in simulator.
   • Frank在设备和模拟器上运行，而iCuke只在模拟器上运行。
   • Frank seems to have a more comprehensive set of step definitions than iCuke's step definitions. And, Frank also has a step definition compendium on their wiki.
   • 与iCuke的步骤定义相比，Frank似乎有更全面的步骤定义。弗兰克在他们的维基上也有一个步骤定义纲要。
   • I proposed that we merge iCuke & Frank (similar to how Merb & Rails merged) since they have the same common goal: Cucumber for iOS.
   • 我建议我们合并iCuke & Frank(类似于Merb和Rails的合并)，因为它们有相同的共同目标:iOS的Cucumber。

2. KIF (Keep It Functional) by Square

   KIF(保持它的功能)是正方形的

3. Zucchini Framework uses Cucumber syntax for writing tests and uses CoffeeScript for step definitions.

   Zucchini框架使用Cucumber语法编写测试，使用CoffeeScript处理步骤定义。

Additions

• OCMock for mocking
• OCMock的嘲笑
• OCHamcrest and/or Expecta for expectations
• OCHamcrest和/或期望的期望。

Conclusion

Well, obviously, there's no right answer to this question, but here's what I'm choosing to go with currently:

显然，这个问题没有正确的答案，但我现在选择的是:

For unit testing, I used to use OCUnit/SenTestingKit in XCode 4. It's simple & solid. But, I prefer the language of BDD over TDD (Why is RSpec better than Test::Unit?) because our words create our world. So now, I use Kiwi with ARC & Kiwi code completion/autocompletion. I prefer Kiwi over Cedar because it's built on top of OCUnit and comes with RSpec-style matchers & mocks/stubs. UPDATE: I'm now looking into OCMock because, currently, Kiwi doesn't support stubbing toll-free bridged objects.

对于单元测试，我曾经在XCode 4中使用OCUnit/SenTestingKit。它的简单和固体。但是，比起TDD，我更喜欢BDD的语言(为什么RSpec比Test更好::Unit?)因为我们的语言创造了我们的世界。现在，我使用Kiwi和ARC & Kiwi代码完成/自动完成。比起雪松，我更喜欢猕猴桃，因为它建在OCUnit的顶部，并配有rspecstyle式的马球、模拟球/存根。更新:我现在正在研究OCMock，因为目前Kiwi不支持存根化的免费桥接对象。

For acceptance testing, I use UI Automation because it's awesome. It lets you record each test case, making writing tests automatic. Also, Apple develops it, and so it has a promising future. It also works on the device and from Instruments, which allows for other cool features, like showing memory leaks. Unfortunately, with UI Automation, I don't know how to run Objective-C code, but with Frank & iCuke you can. So, I'll just test the lower-level Objective-C stuff with unit tests, or create UIButtons only for the TEST build configuration, which when clicked, will run Objective-C code.

对于验收测试，我使用UI自动化，因为它非常棒。它允许您记录每个测试用例，使编写测试自动化。而且，苹果开发了它，所以它有很好的前景。它还可以在设备和仪器上使用，这些设备支持其他很酷的功能，比如显示内存泄漏。不幸的是，有了UI自动化，我不知道如何运行Objective-C代码，但是有了Frank & iCuke，就可以了。因此，我将用单元测试来测试低级的Objective-C，或者为测试构建配置创建uibutton，当点击时，它将运行Objective-C代码。

Which solutions do you use?

你用什么解决方案?

Related Questions

• Is there a BDD solution that presently works well with iOS4 and Xcode4?
• 是否有一种BDD解决方案目前在iOS4和Xcode4上运行良好?
• SenTestingKit (integrated with XCode) versus GHUnit on XCode 4 for Unit Testing?
• SenTestingKit(与XCode集成)与XCode 4中的GHUnit进行单元测试?
• Testing asynchronous code on iOS with OCunit
• 用OCunit测试iOS的异步代码。
• SenTestingKit in Xcode 4: Asynchronous testing?
• Xcode 4中的SenTestingKit:异步测试?
• How does unit testing on the iPhone work?
• iPhone上的单元测试是如何工作的?

8 个解决方案

#1

tl;dr

At Pivotal we wrote Cedar because we use and love Rspec on our Ruby projects. Cedar isn't meant to replace or compete with OCUnit; it's meant to bring the possibility of BDD-style testing to Objective C, just as Rspec pioneered BDD-style testing in Ruby, but hasn't eliminated Test::Unit. Choosing one or the other is largely a matter of style preferences.

在Pivotal，我们编写了杉木，因为我们在Ruby项目中使用和热爱Rspec。雪松并不打算取代或与OCUnit竞争;它旨在将bddstyle测试的可能性引入Objective C，就像Rspec在Ruby中率先使用bddstyle测试一样，但并没有消除Test::Unit。选择其中一个或另一个在很大程度上是风格偏好的问题。

In some cases we designed Cedar to overcome some shortcomings in the way OCUnit works for us. Specifically, we wanted to be able to use the debugger in tests, to run tests from the command line and in CI builds, and get useful text output of test results. These things may be more or less useful to you.

在某些情况下，我们设计雪松来克服OCUnit在我们工作中的一些缺点。具体地说，我们希望能够在测试中使用调试器，在命令行和CI构建中运行测试，并获得测试结果的有用文本输出。这些东西或多或少对你有用。

Long answer

Deciding between two testing frameworks like Cedar and OCUnit (for example) comes down to two things: preferred style, and ease of use. I'll start with the style, because that's simply a matter of opinion and preference; ease of use tends to be a set of tradeoffs.

在雪松和OCUnit(例如)这两种测试框架之间进行选择主要取决于两件事:首选样式和易用性。我将从风格开始，因为这只是一个观点和偏好的问题;易用性往往是一组权衡。

Style considerations transcend what technology or language you use. xUnit-style unit testing has been around for far longer than BDD-style testing, but the latter has rapidly gained in popularity, largely due to Rspec.

风格考虑超越了您使用的技术或语言。与bdd风格的测试相比，xunit式的单元测试已经存在了很长一段时间，但是后者由于Rspec而迅速流行起来。

The primary advantage of xUnit-style testing is its simplicity, and wide adoption (amongst developers who write unit tests); nearly any language you could consider writing code in has an xUnit-style framework available.

xunit式测试的主要优点是它的简单性和广泛的应用(在编写单元测试的开发人员中);几乎任何一种您可以考虑编写代码的语言都具有可以使用的xunit样式框架。

BDD-style frameworks tend to have two main differences when compared to xUnit-style: how you structure the test (or specs), and the syntax for writing your assertions. For me, the structural difference is the main differentiator. xUnit tests are one-dimensional, with one setUp method for all tests in a given test class. The classes that we test, however, aren't one-dimensional; we often need to test actions in several different, potentially conflicting, contexts. For example, consider a simple ShoppingCart class, with an addItem: method (for the purposes of this answer I'll use Objective C syntax). The behavior of this method may differ when the cart is empty compared to when the cart contains other items; it may differ if the user has entered a discount code; it may differ if the specified item can't be shipped by the selected shipping method; etc. As these possible conditions intersect with one another you end up with a geometrically increasing number of possible contexts; in xUnit-style testing this often leads to a lot of methods with names like testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies. The structure of BDD-style frameworks allows you to organize these conditions individually, which I find makes it easier to make sure I cover all cases, as well as easier to find, change, or add individual conditions. As an example, using Cedar syntax, the method above would look like this:

与xunit风格相比，bdd风格的框架有两个主要的区别:如何构造测试(或规范)，以及编写断言的语法。对我来说，结构上的差异是主要的差异。xUnit测试是一维的，对于给定的测试类中的所有测试都有一个设置方法。然而，我们测试的类不是一维的;我们经常需要在几个不同的、可能相互冲突的上下文中测试操作。例如，考虑一个简单的ShoppingCart类，它有一个addItem:方法(对于这个答案，我将使用Objective - C语法)。当购物车为空时，与购物车包含其他项目时相比，此方法的行为可能有所不同;如果用户输入了折扣码，可能会有所不同;如果指定的产品不能通过所选的发货方式发货，则可能会有所不同;当这些可能的条件相交时，你最终会得到一个几何上越来越多的可能的上下文;在xunit样式的测试中，这通常会导致许多方法使用testadditemwhen cartisemptyand nodiscountcodeandshippingmethodapply。bddstyle框架的结构允许您单独组织这些条件，我发现这使我更容易确保涵盖所有情况，也更容易发现、更改或添加个别条件。例如，使用杉木语法，上面的方法是这样的:

describe(@"ShoppingCart", ^{
    describe(@"addItem:", ^{
        describe(@"when the cart is empty", ^{
            describe(@"with no discount code", ^{
                describe(@"when the shipping method applies to the item", ^{
                    it(@"should add the item to the cart", ^{
                        ...
                    });

                    it(@"should add the full price of the item to the overall price", ^{
                        ...
                    });
                });

                describe(@"when the shipping method does not apply to the item", ^{
                    ...
                });
            });

            describe(@"with a discount code", ^{
                ...
            });
        });

        describe(@"when the cart contains other items, ^{
            ...
        });
    });
});

In some cases you'll find contexts in that contain the same sets of assertions, which you can DRY up using shared example contexts.

在某些情况下，您将发现包含相同断言集的上下文，您可以使用共享的示例上下文使其枯竭。

The second main difference between BDD-style frameworks and xUnit-style frameworks, assertion (or "matcher") syntax, simply makes the style of the specs somewhat nicer; some people really like it, others don't.

bddstyle框架和xunit风格框架的第二个主要区别是断言(或“matcher”)语法，这只会让spec的风格更美观一些;有些人真的喜欢，有些人不喜欢。

That leads to the question of ease of use. In this case, each framework has its pros and cons:

这就引出了易用性的问题。在这种情况下，每个框架都有其优缺点:

• OCUnit has been around much longer than Cedar, and is integrated directly into Xcode. This means it's simple to make a new test target, and, most of the time, getting tests up and running "just works." On the other hand, we found that in some cases, such as running on an iOS device, getting OCUnit tests to work was nigh impossible. Setting up Cedar specs takes some more work than OCUnit tests, since you have get the library and link against it yourself (never a trivial task in Xcode). We're working on making setup easier, and any suggestions are more than welcome.

  OCUnit比Cedar要早得多，并且直接集成到Xcode中。这意味着创建一个新的测试目标很简单，而且，大多数时候，让测试启动并运行“只是工作而已”。另一方面，我们发现在某些情况下，比如在iOS设备上运行，让OCUnit测试工作几乎是不可能的。设置Cedar specs需要比OCUnit测试更多的工作，因为您已经获得了库和自己的链接(在Xcode中从来都不是一个简单的任务)。我们正在努力使安装变得更容易，任何建议都非常受欢迎。

• OCUnit runs tests as part of the build. This means you don't need to run an executable to make your tests run; if any tests fail, your build fails. This makes the process of running tests one step simpler, and test output goes directly into your build output window which makes it easy to see. We chose to have Cedar specs build into an executable which you run separately for a few reasons:

  OCUnit运行测试作为构建的一部分。这意味着您不需要运行可执行文件来运行测试;如果任何测试失败，则构建失败。这使得运行测试的过程更简单了一步，并且测试输出直接进入到构建输出窗口中，这样就很容易看到。我们选择将Cedar spec构建为可执行文件，您可以根据以下原因分别运行:

  • We wanted to be able to use the debugger. You run Cedar specs just like you would run any other executable, so you can use the debugger in the same way.
  • 我们希望能够使用调试器。您可以像运行其他可执行文件一样运行Cedar specs，因此您可以以同样的方式使用调试器。
  • We wanted easy console logging in tests. You can use NSLog() in OCUnit tests, but the output goes into the build window where you have to unfold the build step in order to read it.
  • 我们希望在测试中使用简单的控制台登录。您可以在OCUnit测试中使用NSLog()，但是输出会进入构建窗口，您必须展开构建步骤才能读取它。
  • We wanted easy to read test reporting, both on the command line and in Xcode. OCUnit results appear nicely in the build window in Xcode, but building from the command line (or as part of a CI process) results in test output intermingled with lots and lots of other build output. With separate build and run phases Cedar separates the output so the test output is easy to find. The default Cedar test runner copies the standard style of printing "." for each passing spec, "F" for failing specs, etc. Cedar also has the ability to use custom reporter objects, so you can have it output results any way you like, with a little effort.
  • 我们希望在命令行和Xcode中轻松地读取测试报告。OCUnit结果在Xcode中的构建窗口中显示得很好，但是从命令行(或作为CI过程的一部分)构建结果是测试输出与许多其他构建输出混杂在一起。使用独立的构建和运行阶段，杉木将输出分离开来，因此很容易找到测试输出。默认的Cedar test runner复制了标准的打印样式“.”对于每个通过的规格，“F”对于失败的规格等等。

• OCUnit is the official unit testing framework for Objective C, and is supported by Apple. Apple has basically limitless resources, so if they want something done it will get done. And, after all, this is Apple's sandbox we're playing in. The flip side of that coin, however, is that Apple receives on the order of a bajillion support requests and bug reports each day. They're remarkably good about handling them all, but they may not be able to handle issues you report immediately, or at all. Cedar is much newer and less baked than OCUnit, but if you have questions or problems or suggestions send a message to the Cedar mailing list ([e-mail]) and we'll do what we can to help you out. Also, feel free to fork the code from Github (github.com/pivotal/cedar) and add whatever you think is missing. We make our testing frameworks open source for a reason.

  OCUnit是Objective C的官方单元测试框架，由苹果支持。苹果基本上拥有无限的资源，所以如果他们想做点什么，那就去做。毕竟，这是我们在玩的苹果沙盒。然而，问题的另一面是，苹果每天都会收到大量的支持请求和bug报告。他们非常善于处理所有这些问题，但他们可能无法处理您立即报告的问题，或者根本无法处理。雪松比OCUnit更新更少，但如果您有问题或问题或建议，请向雪松邮件列表发送信息([e-mail])，我们将尽我们所能帮助您解决问题。另外，您也可以从Github (github.com/pivotal/cedar)中提取代码，添加您认为缺少的代码。我们让我们的测试框架开源是有原因的。

• Running OCUnit tests on iOS devices can be difficult. Honestly, I haven't tried this for quite some time, so it may have gotten easier, but the last time I tried I simply couldn't get OCUnit tests for any UIKit functionality to work. When we wrote Cedar we made sure that we could test UIKit-dependent code both on the simulator and on devices.

  在iOS设备上运行OCUnit测试可能会很困难。老实说，我已经有一段时间没有尝试过它了，所以它可能变得更容易了，但是上次我尝试的时候，我只是不能让任何UIKit功能的OCUnit测试工作。当我们编写杉木时，我们确保我们可以在模拟器和设备上测试依赖uikit的代码。

Finally, we wrote Cedar for unit testing, which means it's not really comparable with projects like UISpec. It's been quite a while since I tried using UISpec, but I understood it to be focused primarily on programmatically driving the UI on an iOS device. We specifically decided not to try to have Cedar support these types of specs, since Apple was (at the time) about to announce UIAutomation.

最后，我们为单元测试编写了Cedar，这意味着它与UISpec这样的项目没有可比性。自从我尝试使用UISpec以来已经有一段时间了，但我理解它主要关注于在iOS设备上以编程方式驱动UI。我们特别决定不尝试让雪松支持这类规格，因为苹果(当时)即将宣布UIAutomation。

#2

I'm going to have to toss Frank into the acceptance testing mix. This is a fairly new addition but has worked excellent for me so far. Also, it is actually being actively worked on, unlike icuke and the others.

我将不得不把弗兰克扔进验收测试组合中。这是一个相当新的添加，但是到目前为止对我来说效果非常好。而且，与icuke和其他公司不同，它实际上正在积极地进行研究。

#3

For test driven development, I like to use GHUnit, its a breeze to set up, and works great for debugging too.

对于测试驱动开发，我喜欢使用GHUnit，它的设置非常简单，并且对于调试也非常有用。

#4

Great List!

伟大的列表!

I found another interesting solution for UI testing iOS applications.

我发现了另一个有趣的解决方案，用于对iOS应用程序进行UI测试。

Zucchini Framework

西葫芦框架

It is based on UIAutomation. The framework let you write screen centric scenarios in Cucumber like style. The scenarios can be executed in Simulator and on device from a console (it is CI friendly).

它基于UIAutomation。该框架允许您以类似于黄瓜的方式编写屏幕中心场景。场景可以在模拟器中执行，也可以在控制台的设备上执行(这是CI友好的)。

The assertions are screenshot based. Sounds inflexible, but it gets you nice HTML report, with highlighted screen comparison and you can provide masks which define the regions you want to have pixel exact assertion.

断言是基于屏幕截图的。听起来不灵活，但是它为您提供了漂亮的HTML报告，带有突出显示的屏幕比较，并且您可以提供定义您想要的像素精确断言的区域的遮罩。

Each screen has to be described in CoffeScript and the tool it self is written in ruby. It is kind of polyglott nightmare, but the tool provides a nice abstraction for UIAutomation and when the screens are described it is manageable even for QA person.

每个屏幕都必须在CoffeScript中描述，并且它自己用ruby编写的工具。这有点像polyglott噩梦，但是这个工具为UIAutomation提供了一个很好的抽象，当屏幕被描述时，即使是QA人员也可以管理它。

#5

I would choose iCuke for acceptance tests and Cedar for unit tests. UIAutomation is a step in the right direction for Apple, but the tools need better support for continuous integration; automatically running UIAutomation tests with Instruments is currently not possible, for example.

我会选择iCuke进行验收测试，杉木进行单元测试。UIAutomation对于苹果来说是朝着正确方向迈出的一步，但这些工具需要对持续集成提供更好的支持;例如，用仪器自动运行UIAutomation test目前是不可能的。

#6

GHUnit is good for unit tests; for integration tests, I've used UISpec with some success (github fork here: https://github.com/drync/UISpec), but am looking forward to trying iCuke, since it promises to be a lightweight setup, and you can use the rails testing goodness, like RSpec and Cucumber.

GHUnit适用于单元测试;对于集成测试，我已经成功地使用了UISpec (github fork: https://github.com/drync/UISpec)，但是我希望尝试iCuke，因为它承诺是一个轻量级的设置，您可以使用rails测试的优点，比如RSpec和Cucumber。

#7

I currently use specta for rspec like setups and it's partner (as mentioned above) expecta which has ton's of awesome matching options.

我目前在rspec上使用specta，比如设置，它的合作伙伴(如上所述)expecta，它有很多很棒的匹配选项。

#8

I happen to really like OCDSpec2 but I'm biased, I wrote OCDSpec and contribute to the second.

我碰巧很喜欢OCDSpec2，但我有偏见，我写了OCDSpec，并为第二部分做了贡献。

It's very fast even on iOS, in part because it's built from the ground up rather than being put on top of OCUnit. It has an RSpec/Jasmine syntax as well.

它在iOS上的速度非常快，部分是因为它是从头开始构建的，而不是放在OCUnit之上。它也有RSpec/Jasmine语法。

https://github.com/ericmeyer/ocdspec2

https://github.com/ericmeyer/ocdspec2