866535 - Changes in test finder breaks `__$coverObject` globals

Status: RESOLVED WORKSFORME

(Add-on SDK Graveyard :: General, defect, P3)

Description

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

In or around 879a04f0 ( https://github.com/mozilla/addon-sdk/commits/879a04f0 ), something changed with test discovery that makes it that the `global` var at https://github.com/mozilla/addon-sdk/blob/master/lib/sdk/test/harness.js#L583  no longer runs.

FWICT, in this layout:

sdk/test/harness     <- uses CustomLoader, YES `global`
--> test-main.js              <- found by deprecated/unit-test-finder, no `global`
      - require("./main.js")  <- no global

TO FIX: 
- the harness CustomLoader needs to be used at https://github.com/mozilla/addon-sdk/blob/master/lib/sdk/deprecated/unit-test-finder.js#L58 as well.

If you want to see how this impacts an addon... see https://github.com/gregglind/lotsa_addons/tree/master/covertest 



(Also, of people want to revisit how to do code coverage, I am open to suggestions!)

Comment 1

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

I know this sounds like a P3 on the surface, but is a smoke test for the (perhaps) P1 problem of "working on addons is almost, but not quite, like doing other javascript".

Working with addon code in a regimented way is really hard for these reasons:

1.  JS1.7 means the whole tooling stack needs to be forked.  (Or one limits oneself to JS1.5 code).  I have been tempted to move to CoffeeScript or 1.5 just so can actually have linting and validation

2.  Part of why people believe in Micropilot is that I had 99% code coverage.  That was part of why I believed in it as well!   

3.  I am trying to fix this bug, and not making much headway.  The testing code (sdk/test) and discovery (sdk/deprecated/unit*) code is very tangled.  




Solutions:

In the test code, all loaders everywhere share a 'global' object.  Right now, all the "test-" code shares it, but that is not where the coverage needs to happen :)

Comment 2

[Erik Vold [:erikvold] (please needinfo? me)]

Is this still an issue?

Comment 3

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

We still sort of have test coverage working.  It's super flaky and fragile and uses  `freaktechnik/istanbul-jpm`