The other day we had a hard discussion between different developers and project leads, about code coverage tools and the use of the corresponding reports.
前几天,我们在不同的开发人员和项目负责人之间进行了艰苦的讨论,讨论了代码覆盖工具以及相应报告的使用。
- Do you use code coverage in your projects and if so, why not?
- Is code coverage a fixed part of your builds or continous integration or do you just use it from time to time?
- How do you deal with the numbers derived from the reports?
您是否在项目中使用代码覆盖?如果是,为什么不呢?
代码覆盖是构建或连续集成的固定部分,还是您不时地使用它?
你如何处理从报告中得出的数字?
11 个解决方案
#1
We use code coverage to verify that we aren't missing big parts in our testing efforts. Once a milestone or so we run a full coverage report and spend a few days analyzing the results, adding test coverage for areas we missed.
我们使用代码覆盖来验证我们在测试工作中没有遗漏大部分内容。一旦达到里程碑,我们就会运行完整的覆盖率报告并花几天时间分析结果,为我们错过的区域添加测试覆盖率。
We don't run it every build because I don't know that we would analyze it on a regular enough basis to justify that.
我们不会在每次构建时运行它,因为我不知道我们会定期对它进行分析以证明这一点。
We analyze the reports for large blocks of unhit code. We've found this to be the most efficient use. In the past we would try to hit a particular code coverage target but after some point, the returns become very diminishing. Instead, it's better to use code coverage as a tool to make sure you didn't forget anything.
我们分析了大块unhit代码的报告。我们发现这是最有效的用途。在过去,我们会尝试达到特定的代码覆盖率目标,但在某些时候,回报会变得非常小。相反,最好使用代码覆盖作为工具来确保您不会忘记任何事情。
#2
1) Yes we do use code coverage
1)是的我们确实使用代码覆盖率
2) Yes it is part of the CI build (why wouldn't it be?)
2)是的,它是CI构建的一部分(为什么不是?)
3) The important part - we don't look for 100% coverage. What we do look for is buggy/complex code, that's easy to find from your unit tests, and the Devs/Leads will know the delicate parts of the system. We make sure the coverage of such code areas is good, and increases with time, not decreases as people hack in more fixes without the requisite tests.
3)重要的部分 - 我们不寻求100%的覆盖率。我们所寻找的是有缺陷/复杂的代码,可以从单元测试中轻松找到,而Devs / Leads将了解系统的精细部分。我们确保这些代码区域的覆盖率良好,并随着时间的推移而增加,而不是因为人们在没有必要的测试的情况下修复更多修复。
#3
Code coverage tells you how big your "bug catching" net is, but it doesn't tell you how big the holes are in your net.
代码覆盖率告诉您“漏洞”网络有多大,但它并不能告诉您网络中的漏洞有多大。
Use it as an indicator to gauge your testing efforts but not as an absolute metric.
使用它作为衡量您的测试工作的指标,但不是绝对指标。
It is possible to write code that will give you 100% coverage and does not test anything at all.
可以编写能够100%覆盖并且根本不测试任何内容的代码。
#4
The way to look at Code Coverage is to see how much is NOT covered and find out why it is not covered. Code coverage simply tells us that the lines of code is being hit when the unit tests are running. It does not tell us that the code works correctly or not. 100% code coverage is a good number but in medium/large projects it is very hard to achieve.
查看代码覆盖率的方法是查看未涵盖的内容并找出未覆盖的原因。代码覆盖只是告诉我们在单元测试运行时会遇到代码行。它没有告诉我们代码是否正常工作。 100%的代码覆盖率是一个很好的数字,但在中型/大型项目中,很难实现。
#5
I like to measure code coverage on any non-trivial project. As has been mentioned, try not to get too caught up in achieving an arbitrary/magical percentage. There are better metrics, such as riskiness based on complexity, coverage by package/namespace, etc.
我喜欢测量任何非平凡项目的代码覆盖率。正如已经提到的那样,尽量不要太过追求实现任意/神奇的百分比。有更好的指标,例如基于复杂性的风险,包/命名空间的覆盖等。
Take a look at this sample Clover dashboard for similar ideas.
请查看此示例Clover仪表板以获得类似的想法。
#6
We do it in a build, and we see that it should not drop below some value, like 85%. I also do automatic Top 10 Largest Not-covered methods, to know what to start covering.
我们在构建中执行它,并且我们看到它不应低于某个值,如85%。我也做自动十大最大未覆盖方法,知道要开始覆盖什么。
#7
Many teams switching to Agile/XP use code coverage as an indirect way of gauging the ROI of their test automation efforts.
许多转向Agile / XP的团队使用代码覆盖率作为衡量其测试自动化工作投资回报率的间接方式。
I think of it as an experiment - there's an hypothesis that "if we start writing unit tests, our code coverage will improve" - and it makes sense to collect the corresponding observation automatically, via CI, report it in a graph etc.
我认为它是一个实验 - 有一个假设,“如果我们开始编写单元测试,我们的代码覆盖率将会提高” - 并且通过CI自动收集相应的观察,在图表中报告等是有意义的。
You use the results to detect rough spots: if the trend toward more coverage levels off at some point, for instance, you might stop to ask what's going on. Perhaps the team has trouble writing tests that are relevant.
您可以使用结果来检测粗糙点:例如,如果在某个时刻出现更多覆盖范围的趋势,您可能会停下来询问发生了什么。也许团队在编写相关的测试时遇到了麻烦。
#8
We use code coverage to assure that we have no major holes in our tests, and it's run nightly in our CI.
我们使用代码覆盖来确保我们的测试中没有任何重大漏洞,并且它在我们的CI中每晚运行。
Since we also have a full set of selenium-web tests that run all the way through the stack we also do an additional coverage trick:
由于我们还有一整套在整个堆栈中运行的selenium-web测试,我们还会做一个额外的覆盖技巧:
We set up the web-application with coverage running. Then we run the full automated test battery of selenium tests. Some of these are smoke tests only.
我们设置了运行覆盖范围的Web应用程序。然后我们运行硒测试的全自动测试电池。其中一些只是烟雾测试。
When the full suite of tests has been run, we can identify suspected dead code simply by looking at the coverage and inspecting code. This is really nice when working on large projects, because you can have big branches of dead code after some time.
当运行全套测试时,我们只需查看覆盖范围并检查代码即可识别可疑的死代码。在处理大型项目时,这非常好,因为在一段时间后你可以拥有大量的死代码。
We don't really have any fixed metrics on how often we do this, but it's all set up to run with a keypress or two.
我们实际上并没有任何固定的指标,但它们都设置为使用一两个按键运行。
#9
We do use code coverage, it is integrated in our nightly build. There are several tools to analyze the coverage data, commonly they report
我们确实使用代码覆盖,它集成在我们的夜间构建中。有几种工具可以分析覆盖率数据,通常它们会报告
- statement coverage
- branch coverage
- MC/DC coverage
We expect to reach + 90% statement and branch coverage. MC/DC coverage on the other hand gives broader sense for test team. For the uncovered code, we expect justification records by the way.
我们预计将达到+ 90%的声明和分支机构覆盖率。另一方面,MC / DC覆盖范围为测试团队提供了更广泛的意义。对于未覆盖的代码,我们希望顺便说一下理由记录。
#10
I find it depends on the code itself. I won't repeat Joel's statements from SO podcast #38, but the upshot is 'try to be pragmatic'.
我发现它取决于代码本身。我不会重复Joel在SO播客#38中的陈述,但结果是“尽量务实”。
Code coverage is great in core elements of the app.
应用程序的核心元素代码覆盖率很高。
I look at the code as a tree of dependency, if the leaves work (e.g. basic UI or code calling a unit tested DAL) and I've tested them when I've developed them, or updated them, there is a large chance they will work, and if there's a bug, then it won't be difficult to find or fix, so the time taken to mock up some tests will probably be time wasted. Yes there is an issue that updates to code they are dependent on may affect them, but again, it's a case by case thing, and unit tests for the code they are dependent on should cover it.
我将代码视为依赖树,如果叶子工作(例如基本UI或代码调用单元测试DAL)并且我在开发它们时测试它们,或者更新它们,那么它们很有可能会有效,如果有错误,那么找到或修复就不难了,所以模拟一些测试所花费的时间可能会浪费时间。是的,有一个问题是更新它们所依赖的代码可能会影响它们,但同样,这是一个案例的事情,它们所依赖的代码的单元测试应该涵盖它。
When it comes to the trunks or branch of the code, yes code coverage of functionality (as opposed to each function), is very important.
当谈到代码的中继或分支时,功能的代码覆盖(与每个功能相对)是非常重要的。
For example, I recently was on a team that built an app that required a bundle of calculations to calculate carbon emissions. I wrote a suite of tests that tested each and every calculation, and in doing so was happy to see that the dependency injection pattern was working fine.
例如,我最近在一个团队中构建了一个应用程序,需要一组计算来计算碳排放量。我写了一套测试来测试每一个计算,并且这样做很高兴看到依赖注入模式工作正常。
Inevitably, due to a government act change, we had to add a parameter to the equations, and all 100+ tests broke.
不可避免的是,由于*行为改变,我们不得不在方程式中添加一个参数,所有100多个测试都破了。
I realised to update them, over and above testing for a typo (which I could test once), I was unit/regression testing mathematics, and ended up spending the time on building another area of the app instead.
我意识到更新它们,除了测试一个错字(我可以测试一次),我是单位/回归测试数学,并最终花时间建立应用程序的另一个区域。
#11
1) Yes we do measure simple node coverage, beacause:
1)是的我们测量简单的节点覆盖范围,因为:
- it is easy to do with our current project* (Rails web app)
- it encourages our developers to write tests (some come from backgrounds where testing was ad-hoc)
我们当前的项目很容易做到*(Rails网络应用程序)
它鼓励我们的开发人员编写测试(有些来自测试是ad-hoc的背景)
2) Code coverage is part of our continuous integration process.
2)代码覆盖是我们持续集成过程的一部分。
3) The numbers from the reports are used to:
3)报告中的数字用于:
- enforce a minimum level of coverage (95% otherwise the build fails)
- find sections of code which should be tested
强制执行最低覆盖率(95%否则构建失败)
找到应该测试的代码段
There are parts of the system where testing is not all that helpful (usually where you need to make use of mock-objects to deal with external systems). But generally having good coverage makes it easier to maintain a project. One knows that fixes or new features do not break existing functionality.
系统的某些部分测试并不是那么有用(通常需要使用模拟对象来处理外部系统)。但通常具有良好的覆盖范围使维护项目变得更容易。人们知道修复或新功能不会破坏现有功能。
*Details for setting up required coverage for Rails: Min Limit 95 Ahead
*设置Rails所需覆盖范围的详细信息:最小限制95
#1
We use code coverage to verify that we aren't missing big parts in our testing efforts. Once a milestone or so we run a full coverage report and spend a few days analyzing the results, adding test coverage for areas we missed.
我们使用代码覆盖来验证我们在测试工作中没有遗漏大部分内容。一旦达到里程碑,我们就会运行完整的覆盖率报告并花几天时间分析结果,为我们错过的区域添加测试覆盖率。
We don't run it every build because I don't know that we would analyze it on a regular enough basis to justify that.
我们不会在每次构建时运行它,因为我不知道我们会定期对它进行分析以证明这一点。
We analyze the reports for large blocks of unhit code. We've found this to be the most efficient use. In the past we would try to hit a particular code coverage target but after some point, the returns become very diminishing. Instead, it's better to use code coverage as a tool to make sure you didn't forget anything.
我们分析了大块unhit代码的报告。我们发现这是最有效的用途。在过去,我们会尝试达到特定的代码覆盖率目标,但在某些时候,回报会变得非常小。相反,最好使用代码覆盖作为工具来确保您不会忘记任何事情。
#2
1) Yes we do use code coverage
1)是的我们确实使用代码覆盖率
2) Yes it is part of the CI build (why wouldn't it be?)
2)是的,它是CI构建的一部分(为什么不是?)
3) The important part - we don't look for 100% coverage. What we do look for is buggy/complex code, that's easy to find from your unit tests, and the Devs/Leads will know the delicate parts of the system. We make sure the coverage of such code areas is good, and increases with time, not decreases as people hack in more fixes without the requisite tests.
3)重要的部分 - 我们不寻求100%的覆盖率。我们所寻找的是有缺陷/复杂的代码,可以从单元测试中轻松找到,而Devs / Leads将了解系统的精细部分。我们确保这些代码区域的覆盖率良好,并随着时间的推移而增加,而不是因为人们在没有必要的测试的情况下修复更多修复。
#3
Code coverage tells you how big your "bug catching" net is, but it doesn't tell you how big the holes are in your net.
代码覆盖率告诉您“漏洞”网络有多大,但它并不能告诉您网络中的漏洞有多大。
Use it as an indicator to gauge your testing efforts but not as an absolute metric.
使用它作为衡量您的测试工作的指标,但不是绝对指标。
It is possible to write code that will give you 100% coverage and does not test anything at all.
可以编写能够100%覆盖并且根本不测试任何内容的代码。
#4
The way to look at Code Coverage is to see how much is NOT covered and find out why it is not covered. Code coverage simply tells us that the lines of code is being hit when the unit tests are running. It does not tell us that the code works correctly or not. 100% code coverage is a good number but in medium/large projects it is very hard to achieve.
查看代码覆盖率的方法是查看未涵盖的内容并找出未覆盖的原因。代码覆盖只是告诉我们在单元测试运行时会遇到代码行。它没有告诉我们代码是否正常工作。 100%的代码覆盖率是一个很好的数字,但在中型/大型项目中,很难实现。
#5
I like to measure code coverage on any non-trivial project. As has been mentioned, try not to get too caught up in achieving an arbitrary/magical percentage. There are better metrics, such as riskiness based on complexity, coverage by package/namespace, etc.
我喜欢测量任何非平凡项目的代码覆盖率。正如已经提到的那样,尽量不要太过追求实现任意/神奇的百分比。有更好的指标,例如基于复杂性的风险,包/命名空间的覆盖等。
Take a look at this sample Clover dashboard for similar ideas.
请查看此示例Clover仪表板以获得类似的想法。
#6
We do it in a build, and we see that it should not drop below some value, like 85%. I also do automatic Top 10 Largest Not-covered methods, to know what to start covering.
我们在构建中执行它,并且我们看到它不应低于某个值,如85%。我也做自动十大最大未覆盖方法,知道要开始覆盖什么。
#7
Many teams switching to Agile/XP use code coverage as an indirect way of gauging the ROI of their test automation efforts.
许多转向Agile / XP的团队使用代码覆盖率作为衡量其测试自动化工作投资回报率的间接方式。
I think of it as an experiment - there's an hypothesis that "if we start writing unit tests, our code coverage will improve" - and it makes sense to collect the corresponding observation automatically, via CI, report it in a graph etc.
我认为它是一个实验 - 有一个假设,“如果我们开始编写单元测试,我们的代码覆盖率将会提高” - 并且通过CI自动收集相应的观察,在图表中报告等是有意义的。
You use the results to detect rough spots: if the trend toward more coverage levels off at some point, for instance, you might stop to ask what's going on. Perhaps the team has trouble writing tests that are relevant.
您可以使用结果来检测粗糙点:例如,如果在某个时刻出现更多覆盖范围的趋势,您可能会停下来询问发生了什么。也许团队在编写相关的测试时遇到了麻烦。
#8
We use code coverage to assure that we have no major holes in our tests, and it's run nightly in our CI.
我们使用代码覆盖来确保我们的测试中没有任何重大漏洞,并且它在我们的CI中每晚运行。
Since we also have a full set of selenium-web tests that run all the way through the stack we also do an additional coverage trick:
由于我们还有一整套在整个堆栈中运行的selenium-web测试,我们还会做一个额外的覆盖技巧:
We set up the web-application with coverage running. Then we run the full automated test battery of selenium tests. Some of these are smoke tests only.
我们设置了运行覆盖范围的Web应用程序。然后我们运行硒测试的全自动测试电池。其中一些只是烟雾测试。
When the full suite of tests has been run, we can identify suspected dead code simply by looking at the coverage and inspecting code. This is really nice when working on large projects, because you can have big branches of dead code after some time.
当运行全套测试时,我们只需查看覆盖范围并检查代码即可识别可疑的死代码。在处理大型项目时,这非常好,因为在一段时间后你可以拥有大量的死代码。
We don't really have any fixed metrics on how often we do this, but it's all set up to run with a keypress or two.
我们实际上并没有任何固定的指标,但它们都设置为使用一两个按键运行。
#9
We do use code coverage, it is integrated in our nightly build. There are several tools to analyze the coverage data, commonly they report
我们确实使用代码覆盖,它集成在我们的夜间构建中。有几种工具可以分析覆盖率数据,通常它们会报告
- statement coverage
- branch coverage
- MC/DC coverage
We expect to reach + 90% statement and branch coverage. MC/DC coverage on the other hand gives broader sense for test team. For the uncovered code, we expect justification records by the way.
我们预计将达到+ 90%的声明和分支机构覆盖率。另一方面,MC / DC覆盖范围为测试团队提供了更广泛的意义。对于未覆盖的代码,我们希望顺便说一下理由记录。
#10
I find it depends on the code itself. I won't repeat Joel's statements from SO podcast #38, but the upshot is 'try to be pragmatic'.
我发现它取决于代码本身。我不会重复Joel在SO播客#38中的陈述,但结果是“尽量务实”。
Code coverage is great in core elements of the app.
应用程序的核心元素代码覆盖率很高。
I look at the code as a tree of dependency, if the leaves work (e.g. basic UI or code calling a unit tested DAL) and I've tested them when I've developed them, or updated them, there is a large chance they will work, and if there's a bug, then it won't be difficult to find or fix, so the time taken to mock up some tests will probably be time wasted. Yes there is an issue that updates to code they are dependent on may affect them, but again, it's a case by case thing, and unit tests for the code they are dependent on should cover it.
我将代码视为依赖树,如果叶子工作(例如基本UI或代码调用单元测试DAL)并且我在开发它们时测试它们,或者更新它们,那么它们很有可能会有效,如果有错误,那么找到或修复就不难了,所以模拟一些测试所花费的时间可能会浪费时间。是的,有一个问题是更新它们所依赖的代码可能会影响它们,但同样,这是一个案例的事情,它们所依赖的代码的单元测试应该涵盖它。
When it comes to the trunks or branch of the code, yes code coverage of functionality (as opposed to each function), is very important.
当谈到代码的中继或分支时,功能的代码覆盖(与每个功能相对)是非常重要的。
For example, I recently was on a team that built an app that required a bundle of calculations to calculate carbon emissions. I wrote a suite of tests that tested each and every calculation, and in doing so was happy to see that the dependency injection pattern was working fine.
例如,我最近在一个团队中构建了一个应用程序,需要一组计算来计算碳排放量。我写了一套测试来测试每一个计算,并且这样做很高兴看到依赖注入模式工作正常。
Inevitably, due to a government act change, we had to add a parameter to the equations, and all 100+ tests broke.
不可避免的是,由于*行为改变,我们不得不在方程式中添加一个参数,所有100多个测试都破了。
I realised to update them, over and above testing for a typo (which I could test once), I was unit/regression testing mathematics, and ended up spending the time on building another area of the app instead.
我意识到更新它们,除了测试一个错字(我可以测试一次),我是单位/回归测试数学,并最终花时间建立应用程序的另一个区域。
#11
1) Yes we do measure simple node coverage, beacause:
1)是的我们测量简单的节点覆盖范围,因为:
- it is easy to do with our current project* (Rails web app)
- it encourages our developers to write tests (some come from backgrounds where testing was ad-hoc)
我们当前的项目很容易做到*(Rails网络应用程序)
它鼓励我们的开发人员编写测试(有些来自测试是ad-hoc的背景)
2) Code coverage is part of our continuous integration process.
2)代码覆盖是我们持续集成过程的一部分。
3) The numbers from the reports are used to:
3)报告中的数字用于:
- enforce a minimum level of coverage (95% otherwise the build fails)
- find sections of code which should be tested
强制执行最低覆盖率(95%否则构建失败)
找到应该测试的代码段
There are parts of the system where testing is not all that helpful (usually where you need to make use of mock-objects to deal with external systems). But generally having good coverage makes it easier to maintain a project. One knows that fixes or new features do not break existing functionality.
系统的某些部分测试并不是那么有用(通常需要使用模拟对象来处理外部系统)。但通常具有良好的覆盖范围使维护项目变得更容易。人们知道修复或新功能不会破坏现有功能。
*Details for setting up required coverage for Rails: Min Limit 95 Ahead
*设置Rails所需覆盖范围的详细信息:最小限制95