Closed Bug 625818 Opened 9 years ago Closed 9 years ago

Check Panorama mochitest test suite coverage

Categories

(Firefox Graveyard :: Panorama, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mitcho, Assigned: mitcho)

References

Details

Attachments

(2 files, 2 obsolete files)

Previously Mardak noted that they did some instrumenting on Weave at some point to see exactly what lines of code (or maybe just functions) were hit during the test suite, in order to see what code paths are never touched. In light of my experience with bug 625443, I would like to do this with Panorama.

Mardak, could you point us to general resources on how best to do this?
philiKON wrote the tool to us the js debugger to track which lines of code were executed as part of running tests.
This is a fine thing to do after b11 ships.
Blocks: 585689
Priority: -- → P3
Attached patch Mochitest js coverage patch (obsolete) — Splinter Review
Panorama test coverage report is here, based on the code in this patch:

http://mitcho.com/code/panorama-coverage/

Note that, with the jsd interruptHook on, the test suite runs ridiculously slowly... on my machine, the mochitest tabview suite took three hours or so.

The good news: we have about 86% code coverage. Some things which we should write tests for and/or remove:
- gPrivateBrowsing getter
- iQClass_position
- Storage.wipe
- GroupItem_arrange_evaluateDropSpace
- setResizable with no value arg
- GroupItems_saveAll
- GroupItems_getBoundingBox
- GroupItems_removeAll
- GroupItems_getNextGroupItemTab case where there's no activeGroupItem
- GroupItems_moveTabToGroupItem with a groupItemId arg
- Item.dropOptions.out, TabItems.dropOptions.drop, TabItems.dropOptions.over, TabItems.dropOptions.out
- TabItem_forceCanvasSize, TabItem_unforceCanvasSize
- TabPriorityQueue_pop, TabPriorityQueue_peek
- Drag_drag where parent is expanded
- Create an orphan tab on double click (is this really not tested?)
- UI_isIdle
- UI_onTabSelect for apptab
- GroupItem's handleKeyDown handling for return and escape keys and UI__setTabViewFrameKeyHandlers's getClosestTabBy and various keypresses
- UI__saveAll
- TabHandlers.onOther
- Utils.assertThrow

I'm going to make a follow-up blog post on the methodology.
Assignee: nobody → mitcho
Status: NEW → ASSIGNED
For reference, much of the code included in the patch above is based on PhiliKON's work on bug 569228.
And now, a blog post on the method so hopefully others can use this tool in the future:

http://mitcho.com/blog/projects/checking-mochitest-test-coverage/

I now deem this bug resolved! :)
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Comment on attachment 521029 [details] [diff] [review]
Mochitest js coverage patch

New patches above obsolete this patch.
They are updated to apply against recent revisions, plus the following changes:

- reporting tools moved to testing/tools rather than tools/
- more 'noise' lines are ignored as they should
- improved output (now looks like http://neonux.com/StyleEditor/coverage/)
- optional $COVERAGE_OUTPUT (file to store coverage stats if not default $OBJDIR/coverage.json)
- optional $COVERAGE_PROJECT and $COVERAGE_VERSION to give more information on the report
Attachment #521029 - Attachment is obsolete: true
Attachment #554853 - Attachment is obsolete: true
Product: Firefox → Firefox Graveyard
You need to log in before you can comment on or make changes to this bug.