如何自动化功能/集成测试和数据库回滚

时间:2021-07-07 03:52:39

In contrast to my previous question, i'll try to give my requirements.

与我之前的问题相比,我会试着给出我的要求。

I am trying to find some framework/methodology/"thing" that would fit the following:

我正在寻找一些框架/方法/“东西”,以符合以下条件:

  • Ability to write an automated test, preferably written in Visual Studio, using C#.
  • 能够使用c#编写自动化测试,最好是在Visual Studio中编写。
  • Test should drive a web browser and interact with SUT just like an user would.
  • 测试应该驱动web浏览器并像用户一样与SUT交互。
  • Test should be able to setup a test scenario in DB.
  • 测试应该能够在DB中设置一个测试场景。
  • Test should be able to assert that user interactions had the expected effect in DB.
  • 测试应该能够断言用户交互在DB中具有预期的效果。
  • After test is completed, it should be able to roll back all changes it made in DB.
  • 测试完成后,它应该能够回滚它在DB中所做的所有更改。

My first attempt was to use NUnit test to drive Selenium (and Watin before that), but i faced a bit of a problem (check the link above) while using TransactionScope to roll back the changes Selenium-driven browser did in the DB.

我的第一次尝试是使用NUnit测试来驱动Selenium(在此之前还有Watin),但是我在使用TransactionScope回滚DB中Selenium驱动的浏览器时遇到了一些问题(查看上面的链接)。

Has anyone done anything like this in the "real world"? I've found some references through Google, but haven't been able to find any concrete examples on how to implement this. There wouldn't be any problems if i'd be doing unit testing. In that case TransactionScope would be quite enough.

在“现实世界”中有人做过类似的事情吗?我已经通过谷歌找到了一些参考资料,但是还没有找到关于如何实现这一点的具体示例。如果我做单元测试就不会有任何问题。那样的话,事务范围就足够了。

Edit: R. Harvey pointed me to this question, which is almost identical to my situation.

编辑:R. Harvey给我指出了这个问题,这个问题与我的情况几乎相同。

However that question is just almost identical. My application is part of a family of services, all of them accessing the same set of database tables. Amount of test data required does not allow for efficient use of drop/create-scripts, so is there some alternate solution for this?

但是这个问题几乎是一样的。我的应用程序是服务家族的一部分,它们都访问同一组数据库表。所需的测试数据量不允许有效地使用drop/create-scripts,因此是否有其他解决方案?

We are using SQL Server 2005, and i'm not very proficient in database magic, so if there's some way to use sql scripting other than drop/create, then that could be an option.

我们使用的是SQL Server 2005,我不太精通数据库魔法,所以如果除了drop/create之外还有其他方法使用SQL脚本,那么这是一个选项。

Edit 2:

编辑2:

Based on the answers and some additional head scratching, we'll go for more lightweight databases for developers to perform unit-, integration- and functional testing. This enables us to use sql-scripts for setting up and tearing down the test.

基于这些答案和一些额外的麻烦,我们将为开发人员提供更轻量级的数据库来执行单元、集成和功能测试。这使我们能够使用sql脚本设置和分解测试。

3 个解决方案

#1


7  

Changes made in a transaction are only visible inside said transaction. Also wrapping the test in a transaction scope (if possible) would make the test behave differently than the real thing in a very critical aspect (transactions).

事务中所做的更改仅在该事务内可见。在事务范围中包装测试(如果可能的话)也会使测试在非常关键的方面(事务)中的行为与真实的事情不同。

It is much better to use a database image that you restore before every test suite. This way after the suite completes and the verification is done, you drop the test database. The next run, during the suite setup, the database is re-created from the saved image in a pristine state ready for testing. Even better would be to have a script that deploys the database from scratch and run that script during suite setup.

最好使用在每个测试套件之前恢复的数据库映像。这样在套件完成和验证完成之后,您就可以删除测试数据库。在接下来的运行中,在suite设置期间,将从保存的映像中重新创建数据库,使其处于可以进行测试的原始状态。更好的做法是使用一个脚本从头部署数据库,并在suite设置期间运行该脚本。

Btw is not feasible to restore to a pristine state before every test. More generically is not feasible to have lengthy individual test setup and cleanup steps. As you add more tests the time spent restoring the database to test-ready condition between tests will become just unmanageable. Suites with hundreds of tests are quite common and full test runs of tens of thousands of tests would mean hours and hours spent just restoring database for test. Design your individual test so that they can be run independently, ie. test N has to produce valid results even if test N-1 failed.

在每次测试前恢复到原始状态是不可行的。更一般化的是不可能有冗长的单独测试设置和清理步骤。随着您添加更多的测试,在测试之间将数据库恢复到测试就绪状态所花费的时间将变得难以管理。有数百个测试的套件是相当常见的,并且测试运行的成千上万个测试将意味着要花几个小时和几个小时来恢复数据库的测试。设计你的个人测试,这样他们就可以独立运行了。即使测试N-1失败,测试N也必须产生有效的结果。

Another thing to consider is failure investigation, you want your failed test to leave the database in a state that can be investigated for meaningful info and you want subsequent tests to be able to run and produce valid results. Sometimes these requirements will contradict each other, but you must take them into consideration and design your test around them.

另一件要考虑的事情是失败调查,您希望失败的测试将数据库保持在可以进行调查以获得有意义的信息的状态,并希望后续的测试能够运行并产生有效的结果。有时这些需求会相互矛盾,但是您必须考虑它们并围绕它们设计您的测试。

#2


7  

If the amount of data required to restore the database to a known-good state is prohibitive of drop/create scripts and you are running you tests on Developer or Enterprise edition of SQL 2005, you could look into creating a database snapshot of the good state, and reverting to it before each test. This is considerably faster than a full restore, although it may still be too time consuming if you have hundreds of tests.

如果需要恢复数据库的数据量好的状态是禁止删除/创建脚本和你运行测试SQL 2005的开发人员或企业版,你可以考虑创建一个数据库快照的状态,并在每个测试之前恢复到它。这比完整的恢复要快得多,尽管如果您有数百个测试,这可能仍然太耗时。

#3


0  

Don't miss Amnesia which I recommended on this related question.

不要错过我在这个相关问题上建议的健忘症。

#1


7  

Changes made in a transaction are only visible inside said transaction. Also wrapping the test in a transaction scope (if possible) would make the test behave differently than the real thing in a very critical aspect (transactions).

事务中所做的更改仅在该事务内可见。在事务范围中包装测试(如果可能的话)也会使测试在非常关键的方面(事务)中的行为与真实的事情不同。

It is much better to use a database image that you restore before every test suite. This way after the suite completes and the verification is done, you drop the test database. The next run, during the suite setup, the database is re-created from the saved image in a pristine state ready for testing. Even better would be to have a script that deploys the database from scratch and run that script during suite setup.

最好使用在每个测试套件之前恢复的数据库映像。这样在套件完成和验证完成之后,您就可以删除测试数据库。在接下来的运行中,在suite设置期间,将从保存的映像中重新创建数据库,使其处于可以进行测试的原始状态。更好的做法是使用一个脚本从头部署数据库,并在suite设置期间运行该脚本。

Btw is not feasible to restore to a pristine state before every test. More generically is not feasible to have lengthy individual test setup and cleanup steps. As you add more tests the time spent restoring the database to test-ready condition between tests will become just unmanageable. Suites with hundreds of tests are quite common and full test runs of tens of thousands of tests would mean hours and hours spent just restoring database for test. Design your individual test so that they can be run independently, ie. test N has to produce valid results even if test N-1 failed.

在每次测试前恢复到原始状态是不可行的。更一般化的是不可能有冗长的单独测试设置和清理步骤。随着您添加更多的测试,在测试之间将数据库恢复到测试就绪状态所花费的时间将变得难以管理。有数百个测试的套件是相当常见的,并且测试运行的成千上万个测试将意味着要花几个小时和几个小时来恢复数据库的测试。设计你的个人测试,这样他们就可以独立运行了。即使测试N-1失败,测试N也必须产生有效的结果。

Another thing to consider is failure investigation, you want your failed test to leave the database in a state that can be investigated for meaningful info and you want subsequent tests to be able to run and produce valid results. Sometimes these requirements will contradict each other, but you must take them into consideration and design your test around them.

另一件要考虑的事情是失败调查,您希望失败的测试将数据库保持在可以进行调查以获得有意义的信息的状态,并希望后续的测试能够运行并产生有效的结果。有时这些需求会相互矛盾,但是您必须考虑它们并围绕它们设计您的测试。

#2


7  

If the amount of data required to restore the database to a known-good state is prohibitive of drop/create scripts and you are running you tests on Developer or Enterprise edition of SQL 2005, you could look into creating a database snapshot of the good state, and reverting to it before each test. This is considerably faster than a full restore, although it may still be too time consuming if you have hundreds of tests.

如果需要恢复数据库的数据量好的状态是禁止删除/创建脚本和你运行测试SQL 2005的开发人员或企业版,你可以考虑创建一个数据库快照的状态,并在每个测试之前恢复到它。这比完整的恢复要快得多,尽管如果您有数百个测试,这可能仍然太耗时。

#3


0  

Don't miss Amnesia which I recommended on this related question.

不要错过我在这个相关问题上建议的健忘症。