Open Bug 750347 Opened 7 years ago Updated 2 years ago

[meta] Record JavaScript code coverage of automated tests

Categories

(Testing :: Code Coverage, defect)

defect
Not set

Tracking

(Not tracked)

People

(Reporter: gps, Unassigned)

References

(Depends on 1 open bug)

Details

It would be splendid if the build system measured and recorded code coverage of the in-tree JavaScript tests. This includes xpcshell and mochitests.

Some people have hacked this up before. I'd like to see it officially integrated with the build system as a build target (e.g. `make xpcshell-test-coverage`) and eventually have the results recorded by the TBPL build infrastructure.

This bug will be a meta/tracking bug. Technical discussion will likely occur in child bugs.
Depends on: 750364
Depends on: 1204554
No longer depends on: 750364
Depends on: 1210733
I have some results on running code coverage on comm-central's mailnews xpcshell tests here:
<https://www.tjhsst.edu/~jcranmer/test-ui/coverage.html?dir=mailnews> (selecting mailnews here because a lot of the mozilla code wasn't tested/run). I can already tell you by looking at that coverage that there appears to be one xpcshell test which is broken (!).

The necessary infrastructure is:
1. A tool to dump a map of runtime URLs to source file locations in the tree
2. A tool to post-process the output LCOV files to remap file locations as mentioned above
3. A tool to calculate the uncoverage of code not run.

I have prototypes of all of these pieces that I used to build the aforementioned site.
there are 2 ways to record this:
1) emitting lcov output from the JSVM, there is some work here with a small list of known bugs remaining (linux64-ccov)
2) using devtools debugging api to get coverage, this works in general, but there are a small list of remaining issues (linux64-jsdcov)

using jsdcov we can capture per test info easier, but we lack some of the deeper collection of coverage.

I would like to know what we want to use this bug to track (or duplicate it to).  if this is JSVM coverage, we have a tracking bug for that (bug 1189360) and if this is jsdcov, we have a tracking bug for that (bug 1301174).
Flags: needinfo?(gps)
I have no clue. Maybe we should make bug 1189360 and bug 1301174 block this one, effectively turning this bug into a master tracker and those other bugs into trackers for the tech-specific coverage mechanism?
Flags: needinfo?(gps)
You need to log in before you can comment on or make changes to this bug.