cfx test should support a --coverage option



6 years ago
5 years ago


(Reporter: glind, Unassigned)


Firefox Tracking Flags

(Not tracked)


The meat of it seems to be:

* change `require`, such that if it's a file we care about, it gets instrumented.
* read in the file, parse it using esprima, instrument it (, and remake it using escodegen.  takes a different approach:  here you copy in 'instrumented' versions of the code, then just run your tests on that.  Since we are copying around files anyway, perhaps this is an attractive option?

This is a lot of work to implement, but would help *a lot*.  Plus, coverage reports look cool.
Escodegen and esparsa don't know the full moz, dialect, notably things like 

   const {a,b,c} = {}  /*  "ObjectPattern" */

A fine solution would be to push those changes up into Escodegen and Esparsa.  Spidermonkey's "Reflect.parse()" gets these right, but it would be nicer to have a pure js solution.
So far this has been quite tedious to work on :)  

Neither Esprima nor escodegen have fully finished ports.  I have a very sad face.

CoverJS is very promising, as a runner.
Priority: -- → P3
I am pushing all the code up into esprima#harmony.  I have been running my changes over the addon-sdk code to find warts as well, and would love to share results.  In the meanwhile, I am happy to own this.
Most of the missing features have gone upstream into esprima, and for the others, I am maintaining.  I am working on getting escodegen up to snuff, but I am still not sure of the best way to get the actual instrumentation to happen.

Now I am deep in the bowels of cfx, harness-options.js, packages/api-utils/lib/addon/runner.js and the like.... where can one actually hook in here, to either change Require, or keep track of some global object to keep the counts?

Comment 8

6 years ago
Commits pushed to master at
CoverObject, work in progress (bug 789401)

Discussion of method and approaches:

Recipe for usage at:
Merge pull request #624 from gregglind/coverage

Bug 789401 - CoverObject, work in progress r=@gozala
I'm going through the list of open bugs that github robot has commented in. Is this bug fixed, Erik?
Flags: needinfo?(evold)
(In reply to Wes Kocher (:KWierso) from comment #9)
> I'm going through the list of open bugs that github robot has commented in.
> Is this bug fixed, Erik?

Doesn't look like it.
Flags: needinfo?(evold)
My last attempt on this failed to make much progress.  Before, it relied on global 'coverage' object that all tests could see.  Alternative paths on this are welcome!
I want to chat about this at triage this week, we have an number of pull requests open for this and we need to figure out what to do with them.

Priority: P3 → --
as requested
Flags: needinfo?(dtownsend+bugmail)
At this point we don't have a decision on how to proceed and this isn't important enough for the SDK team to spend time on compared to other work. I'm going to close this. If the way forward becomes clearer in the future then we can reopen and talk about what taking a patch would look like.
Flags: needinfo?(dtownsend+bugmail)
Last Resolved: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.