cfx test should support a --coverage option

RESOLVED WONTFIX

Status

Add-on SDK
General
RESOLVED WONTFIX
5 years ago
3 years ago

People

(Reporter: Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat ), Unassigned)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

possible ways in:

http://esprima.org/doc/#unittests
https://github.com/itay/node-cover#readme
The meat of it seems to be:

* change `require`, such that if it's a file we care about, it gets instrumented.
  (https://github.com/itay/node-cover/blob/master/index.js#L343)
* read in the file, parse it using esprima, instrument it (https://github.com/itay/node-cover/blob/master/instrument.js#L111), and remake it using escodegen.


https://github.com/arian/CoverJS  takes a different approach:  here you copy in 'instrumented' versions of the code, then just run your tests on that.  Since we are copying around files anyway, perhaps this is an attractive option?

This is a lot of work to implement, but would help *a lot*.  Plus, coverage reports look cool.
Escodegen and esparsa don't know the full moz, dialect, notably things like 

   const {a,b,c} = {}  /*  "ObjectPattern" */

A fine solution would be to push those changes up into Escodegen and Esparsa.  Spidermonkey's "Reflect.parse()" gets these right, but it would be nicer to have a pure js solution.
So far this has been quite tedious to work on :)  

Neither Esprima nor escodegen have fully finished ports.  I have a very sad face.

CoverJS is very promising, as a runner.

Updated

5 years ago
Priority: -- → P3
I am pushing all the code up into esprima#harmony.  I have been running my changes over the addon-sdk code to find warts as well, and would love to share results.  In the meanwhile, I am happy to own this.
Most of the missing features have gone upstream into esprima, and for the others, I am maintaining.  I am working on getting escodegen up to snuff, but I am still not sure of the best way to get the actual instrumentation to happen.

GL
Now I am deep in the bowels of cfx, harness-options.js, packages/api-utils/lib/addon/runner.js and the like.... where can one actually hook in here, to either change Require, or keep track of some global object to keep the counts?
Progress report:  https://gist.github.com/3900355

Comment 8

4 years ago
Commits pushed to master at https://github.com/mozilla/addon-sdk

https://github.com/mozilla/addon-sdk/commit/fa2458dfa1821ab75f58519088a73cd055a30b17
CoverObject, work in progress (bug 789401)

Discussion of method and approaches:

    https://bugzilla.mozilla.org/show_bug.cgi?id=789401

Recipe for usage at:

    https://gist.github.com/3900355

https://github.com/mozilla/addon-sdk/commit/2b94404eec571cafcc3c856caffaa4b29c5b52eb
Merge pull request #624 from gregglind/coverage

Bug 789401 - CoverObject, work in progress r=@gozala
Blocks: 840023
I'm going through the list of open bugs that github robot has commented in. Is this bug fixed, Erik?
Flags: needinfo?(evold)
(In reply to Wes Kocher (:KWierso) from comment #9)
> I'm going through the list of open bugs that github robot has commented in.
> Is this bug fixed, Erik?

Doesn't look like it.
Flags: needinfo?(evold)
My last attempt on this failed to make much progress.  Before, it relied on global 'coverage' object that all tests could see.  Alternative paths on this are welcome!
I want to chat about this at triage this week, we have an number of pull requests open for this and we need to figure out what to do with them.

Dave
Priority: P3 → --
as requested
Flags: needinfo?(dtownsend+bugmail)
At this point we don't have a decision on how to proceed and this isn't important enough for the SDK team to spend time on compared to other work. I'm going to close this. If the way forward becomes clearer in the future then we can reopen and talk about what taking a patch would look like.
Flags: needinfo?(dtownsend+bugmail)
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.