I understand the short answer to it is by doing testing but still how do you do this testing? Do you modify your test cases to include the bugs as additional test cases to run or do you just verify all the bugs in your bug tracking system starting from the oldest to the newest releases.
我理解简短的答案是通过测试,但你仍然如何进行测试?您是否修改了测试用例以将错误包含在运行的其他测试用例中,或者您只是验证错误跟踪系统中从最旧版本到最新版本的所有错误。
Thanks for the answers. Looks like my question was not clear. I understand we need to write bug report, fix the bug & do the testing for fix verification. However, under which test phase should this test go so that during the next version release, we are sure to re-run the test again to make sure none of the new changes have re-introduced the bug. Should it go under regression testing or should it go under integration testing for that specific project or should we just test all the bugs in the bug tracking system since version 1.0?
谢谢你的回答。看起来我的问题不明确。我知道我们需要编写错误报告,修复错误并进行修复验证测试。但是,在下一个版本发布期间,测试阶段应该在下一个版本发布期间,我们确保再次重新运行测试以确保没有新的更改重新引入该错误。它是应该进行回归测试还是应该针对该特定项目进行集成测试,还是应该从1.0版开始测试bug跟踪系统中的所有错误?
7 个解决方案
#1
What do you mean by saying "You're protected from regressing it, because that test will suddenly fail."?
你是什么意思说“你被保护不会退回它,因为那个测试会突然失败。”?
The point of "unit tests" is that they will be run automatically as part of the compile/check cycle: you'll run the tests after compiling and before checking code in. If a unit test fails, then you know that some code that you just wrote will recreate the old bug.
“单元测试”的意思是它们将作为编译/检查周期的一部分自动运行:您将在编译之后和检查代码之前运行测试。如果单元测试失败,那么您知道某些代码你刚写的将重新创建旧的bug。
edit:
How can you be so sure that just because your unit tests passed all your bugs have been fixed.
你怎么能这么肯定,因为你的单元测试通过所有你的错误已经修复。
Generally, if you can reproduce a bug, you can make a test to duplicate it in code. The most important part is once you've written a test to check for the existance of a bug, then if you run the tests, and the test fails, then you reintroduced the bug. One great book on the subject is "Working Effectively with Legacy Code"
通常,如果您可以重现错误,则可以进行测试以在代码中复制它。最重要的部分是,一旦你编写了一个测试来检查bug的存在,然后如果你运行测试,并且测试失败,那么你重新引入了bug。关于这个主题的一本好书是“有效地使用遗留代码”
Is it not possible for a single bug to span multiple unit tests.
单个错误是否不可能跨越多个单元测试。
Of course it can span multiple tests.
当然它可以跨越多个测试。
Let's presume the bug is "can't save files."
让我们假设这个bug是“无法保存文件”。
And let's presume that there are several different scenarios that can cause this:
我们假设有几种不同的情况可能导致这种情况:
-
out of disk space,
磁盘空间不足,
-
the file you're writing on top of is opened and locked by another process,
您正在写的文件被另一个进程打开并锁定,
-
there are permissions problems (like you can't access the directory, or don't have write permissions),
有权限问题(比如你无法访问目录,或者没有写权限),
-
or there is a network error while you're saving the file.
或者在保存文件时出现网络错误。
Ideally, your test harness (a collection of tests) would have a separate test for each one. For some of the tests, you may need to use stuff that are called "mock objects" (they can also be used when you haven't written the code for the component yet).
理想情况下,您的测试工具(测试集合)将针对每个测试工具进行单独测试。对于某些测试,您可能需要使用被称为“模拟对象”的东西(当您还没有为组件编写代码时也可以使用它们)。
#2
You create an automated test that reproduces the problem. You then fix the problem, and make sure that the new test and all your existing tests still pass.
您创建了一个可以重现问题的自动化测试。然后,您可以解决问题,并确保新测试和所有现有测试仍然通过。
This should be regression testing, and should be automated to the extent possible.
这应该是回归测试,并且应尽可能自动化。
#3
When you find a bug, the first thing you need to do is make a unit test that demonstrates the bug. You'll know when you've fixed it, because the test passes. You're protected from regressing it, because that test will suddenly fail.
当您发现错误时,您需要做的第一件事是进行单元测试以演示错误。你知道什么时候修好它,因为测试通过了。你被保护不会退回它,因为那个测试会突然失败。
#4
- Get a bug report
- Write test case to reproduce bug report
- Fix bug
- Enjoy not having to fix that bug again
获取错误报告
编写测试用例来重现错误报告
享受不必再次修复该bug
#5
A "bug" in a system is just some unit of functionality not working. By createing unit tests that covers the unit of functionality you can know if the bug reappears for that unit of work. Now if you write additional unit tests testing other areas of functionality that might fail in a simular situation you will get more coverage.
系统中的“错误”只是某些功能单元无法正常工作。通过创建涵盖功能单元的单元测试,您可以了解该工作单元是否再次出现该错误。现在,如果您编写额外的单元测试来测试在类似情况下可能会失败的其他功能区域,您将获得更多的覆盖范围。
That said unit testing is only part of the answer. Adding a test case for this bug as well will also help catch it from occuring but unfortunatly it will be later in the development lifecycle and more expensive to fix.
那说单元测试只是答案的一部分。为这个bug添加一个测试用例也有助于防止它发生,但不幸的是它将在开发生命周期的后期修复并且更加昂贵。
#6
As a QA you need to anticipate the areas that would get affected as a result of some fixes. Thorougly test the application for the same though it is highly recommended to regress the entire build after each freeze.
作为质量保证,您需要预测因某些修复而受到影响的区域。尽管强烈建议在每次冻结后对整个构建进行回归,但Thorougly会测试相同的应用程序。
If it is a long term project and you have time at hand, then it is a good practice to automate the test however if the project is being developed in agile model within short span of time and changing requirement then do it with manual testing trying to cover as much as possible.
如果它是一个长期项目并且您有时间,那么自动化测试是一个很好的做法,但是如果项目是在短时间内以敏捷模型开发并且需要更改,那么请尝试手动测试尽可能覆盖。
#7
Depending from which perspective you ask, you can attack issue in different ways. From developers perspective, it could be best to have all bugs since 1.0 (or earlier) checked with proper unit test, and let it run even with CI. When it becomes to much hassle try to group it, essential tests that will run with every build, some once a day, rest once a week (or something).
根据您提出的观点,您可以通过不同方式攻击问题。从开发人员的角度来看,最好是通过适当的单元测试检查1.0(或更早)的所有错误,并让它在CI中运行。当它变得非常麻烦时尝试将它分组,每次构建运行的基本测试,每天一次,每周休息一次(或某事)。
From QA/Testing perspective it requires a bit more effort.
从质量保证/测试的角度来看,它需要更多的努力。
- Prepare regression suit for application. Main business processes/use cases, the happy paths. Make it your baseline. Set of tests that must fail under no circumstances. Run it always before release or when big changes are made, when many changes are made, when critical parts are modified. Put some effort to automate it (and this time I'm not talking about unit tests, but full application tests).
- Than create regression tests suits for areas of the application. More tests than in previous suit, but each regression suit focusing on different application area. When you change one part of application, run regression for that part. it can be automated, but it can be hard to maintain tests for areas that change often.
- Educate your qa team to look through bugs repository as they start working on tests to some requirement/functionality. They should check for previous bugs if they seem relevant.
准备回归诉讼申请。主要业务流程/用例,快乐路径。把它作为你的基准。在任何情况下都必须失败的测试集。在发布之前或进行大的更改时,在进行许多更改时,在修改关键部件时始终运行它。付出一些努力使其自动化(这次我不是在讨论单元测试,而是完整的应用程序测试)。
比创建回归测试适合应用领域。比以前的西装更多的测试,但每个回归适合不同的应用领域。更改应用程序的一部分时,请对该部分运行回归。它可以自动化,但是对于经常变化的区域维护测试可能很困难。
教育你的qa团队在他们开始对一些需求/功能进行测试时查看bug库。如果它们看似相关,它们应该检查以前的错误。
Keep in mind that you must be careful with regression testing, test automation, and desire to check for all bugs from version 1.0. Maybe application changed so much since version 1.0 that some bugs are no longer relevant? That should be taken into consideration when building regression tests based on old bugs.
请记住,您必须小心使用回归测试,测试自动化以及检查版本1.0中的所有错误。也许应用程序自版本1.0以来发生了很大变化,以至于某些错误不再相关?在基于旧错误构建回归测试时应该考虑这一点。
#1
What do you mean by saying "You're protected from regressing it, because that test will suddenly fail."?
你是什么意思说“你被保护不会退回它,因为那个测试会突然失败。”?
The point of "unit tests" is that they will be run automatically as part of the compile/check cycle: you'll run the tests after compiling and before checking code in. If a unit test fails, then you know that some code that you just wrote will recreate the old bug.
“单元测试”的意思是它们将作为编译/检查周期的一部分自动运行:您将在编译之后和检查代码之前运行测试。如果单元测试失败,那么您知道某些代码你刚写的将重新创建旧的bug。
edit:
How can you be so sure that just because your unit tests passed all your bugs have been fixed.
你怎么能这么肯定,因为你的单元测试通过所有你的错误已经修复。
Generally, if you can reproduce a bug, you can make a test to duplicate it in code. The most important part is once you've written a test to check for the existance of a bug, then if you run the tests, and the test fails, then you reintroduced the bug. One great book on the subject is "Working Effectively with Legacy Code"
通常,如果您可以重现错误,则可以进行测试以在代码中复制它。最重要的部分是,一旦你编写了一个测试来检查bug的存在,然后如果你运行测试,并且测试失败,那么你重新引入了bug。关于这个主题的一本好书是“有效地使用遗留代码”
Is it not possible for a single bug to span multiple unit tests.
单个错误是否不可能跨越多个单元测试。
Of course it can span multiple tests.
当然它可以跨越多个测试。
Let's presume the bug is "can't save files."
让我们假设这个bug是“无法保存文件”。
And let's presume that there are several different scenarios that can cause this:
我们假设有几种不同的情况可能导致这种情况:
-
out of disk space,
磁盘空间不足,
-
the file you're writing on top of is opened and locked by another process,
您正在写的文件被另一个进程打开并锁定,
-
there are permissions problems (like you can't access the directory, or don't have write permissions),
有权限问题(比如你无法访问目录,或者没有写权限),
-
or there is a network error while you're saving the file.
或者在保存文件时出现网络错误。
Ideally, your test harness (a collection of tests) would have a separate test for each one. For some of the tests, you may need to use stuff that are called "mock objects" (they can also be used when you haven't written the code for the component yet).
理想情况下,您的测试工具(测试集合)将针对每个测试工具进行单独测试。对于某些测试,您可能需要使用被称为“模拟对象”的东西(当您还没有为组件编写代码时也可以使用它们)。
#2
You create an automated test that reproduces the problem. You then fix the problem, and make sure that the new test and all your existing tests still pass.
您创建了一个可以重现问题的自动化测试。然后,您可以解决问题,并确保新测试和所有现有测试仍然通过。
This should be regression testing, and should be automated to the extent possible.
这应该是回归测试,并且应尽可能自动化。
#3
When you find a bug, the first thing you need to do is make a unit test that demonstrates the bug. You'll know when you've fixed it, because the test passes. You're protected from regressing it, because that test will suddenly fail.
当您发现错误时,您需要做的第一件事是进行单元测试以演示错误。你知道什么时候修好它,因为测试通过了。你被保护不会退回它,因为那个测试会突然失败。
#4
- Get a bug report
- Write test case to reproduce bug report
- Fix bug
- Enjoy not having to fix that bug again
获取错误报告
编写测试用例来重现错误报告
享受不必再次修复该bug
#5
A "bug" in a system is just some unit of functionality not working. By createing unit tests that covers the unit of functionality you can know if the bug reappears for that unit of work. Now if you write additional unit tests testing other areas of functionality that might fail in a simular situation you will get more coverage.
系统中的“错误”只是某些功能单元无法正常工作。通过创建涵盖功能单元的单元测试,您可以了解该工作单元是否再次出现该错误。现在,如果您编写额外的单元测试来测试在类似情况下可能会失败的其他功能区域,您将获得更多的覆盖范围。
That said unit testing is only part of the answer. Adding a test case for this bug as well will also help catch it from occuring but unfortunatly it will be later in the development lifecycle and more expensive to fix.
那说单元测试只是答案的一部分。为这个bug添加一个测试用例也有助于防止它发生,但不幸的是它将在开发生命周期的后期修复并且更加昂贵。
#6
As a QA you need to anticipate the areas that would get affected as a result of some fixes. Thorougly test the application for the same though it is highly recommended to regress the entire build after each freeze.
作为质量保证,您需要预测因某些修复而受到影响的区域。尽管强烈建议在每次冻结后对整个构建进行回归,但Thorougly会测试相同的应用程序。
If it is a long term project and you have time at hand, then it is a good practice to automate the test however if the project is being developed in agile model within short span of time and changing requirement then do it with manual testing trying to cover as much as possible.
如果它是一个长期项目并且您有时间,那么自动化测试是一个很好的做法,但是如果项目是在短时间内以敏捷模型开发并且需要更改,那么请尝试手动测试尽可能覆盖。
#7
Depending from which perspective you ask, you can attack issue in different ways. From developers perspective, it could be best to have all bugs since 1.0 (or earlier) checked with proper unit test, and let it run even with CI. When it becomes to much hassle try to group it, essential tests that will run with every build, some once a day, rest once a week (or something).
根据您提出的观点,您可以通过不同方式攻击问题。从开发人员的角度来看,最好是通过适当的单元测试检查1.0(或更早)的所有错误,并让它在CI中运行。当它变得非常麻烦时尝试将它分组,每次构建运行的基本测试,每天一次,每周休息一次(或某事)。
From QA/Testing perspective it requires a bit more effort.
从质量保证/测试的角度来看,它需要更多的努力。
- Prepare regression suit for application. Main business processes/use cases, the happy paths. Make it your baseline. Set of tests that must fail under no circumstances. Run it always before release or when big changes are made, when many changes are made, when critical parts are modified. Put some effort to automate it (and this time I'm not talking about unit tests, but full application tests).
- Than create regression tests suits for areas of the application. More tests than in previous suit, but each regression suit focusing on different application area. When you change one part of application, run regression for that part. it can be automated, but it can be hard to maintain tests for areas that change often.
- Educate your qa team to look through bugs repository as they start working on tests to some requirement/functionality. They should check for previous bugs if they seem relevant.
准备回归诉讼申请。主要业务流程/用例,快乐路径。把它作为你的基准。在任何情况下都必须失败的测试集。在发布之前或进行大的更改时,在进行许多更改时,在修改关键部件时始终运行它。付出一些努力使其自动化(这次我不是在讨论单元测试,而是完整的应用程序测试)。
比创建回归测试适合应用领域。比以前的西装更多的测试,但每个回归适合不同的应用领域。更改应用程序的一部分时,请对该部分运行回归。它可以自动化,但是对于经常变化的区域维护测试可能很困难。
教育你的qa团队在他们开始对一些需求/功能进行测试时查看bug库。如果它们看似相关,它们应该检查以前的错误。
Keep in mind that you must be careful with regression testing, test automation, and desire to check for all bugs from version 1.0. Maybe application changed so much since version 1.0 that some bugs are no longer relevant? That should be taken into consideration when building regression tests based on old bugs.
请记住,您必须小心使用回归测试,测试自动化以及检查版本1.0中的所有错误。也许应用程序自版本1.0以来发生了很大变化,以至于某些错误不再相关?在基于旧错误构建回归测试时应该考虑这一点。