1350445 - [meta] Generating Code Coverage locally

Status: RESOLVED FIXED

(Testing :: Code Coverage, enhancement)

Description

[Kyle Lahnakoski [:ekyle]]

This is a tracking bug for generating code coverage on developer machines.  This includes, but not limited to

1) Verifying and correcting instructions for generating coverage locally
2) Instructions and overview for using our unified coverage tool(s)

Comment 1

[Marco Castelluccio [:marco]]

The instructions on the MDN page should be up to date: https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Measuring_Code_Coverage_on_Firefox.

Comment 2

[Kyle Lahnakoski [:ekyle]]

Greg,

Please review these instructions, then we can close this bug.

Comment 3

[Greg Mierzwinski [:sparky]]

The JSDCOV part is good and definitely works. I'll try to run code coverage from a one-click loaner, and then locally and let you know if I run into any problems or if everything is good. I'll clear the needinfo flag when I'm done.

Comment 4

[Greg Mierzwinski [:sparky]]

So I've started working on testing everything there. First, I can only test the local instructions, I don't have permission to use one-click loaners when I try.

So far, the instructions are easy to follow but I've run into a problem. I've modified the .mozconfig file to contain everything that is mentioned on the page. They are then read when I run |./mach build|. However, I can't find any gcno's created after this. Furthermore, I ran |./mach mochitest browser/base/content/| and there were no .gcda files created.

I've changed the flags to use --coverage instead to see if that gets me anything. I'll comment here with the results when I get them.

Comment 5

[Greg Mierzwinski [:sparky]]

I've finished reviewing them and made a few small changes to the document. The only thing left to check is that the one-click-loaner instructions work since I can't test that one.

Other than that, I had to change the flags to |--coverage| on my machine (linux mint 18.1 - cinnamon) to get the gcno and gcda files out. I left a note about that on the page. All the other commands work great without any problems.

I think we can close this bug now, unless we want to test the one-click loaner commands (there are only 2, and it looks like they will work).

Comment 6

[Marco Castelluccio [:marco]]

(In reply to Greg Mierzwinski [:gmierz] from comment #5)
> I've finished reviewing them and made a few small changes to the document.
> The only thing left to check is that the one-click-loaner instructions work
> since I can't test that one.
> 
> Other than that, I had to change the flags to |--coverage| on my machine
> (linux mint 18.1 - cinnamon) to get the gcno and gcda files out. I left a
> note about that on the page. All the other commands work great without any
> problems.

To avoid confusion, since "--coverage" works for everyone, we can just remove all references to the other options and always use "--coverage". What do you think?

Comment 7

[Greg Mierzwinski [:sparky]]

I agree, I've changed the document to use those flags. I left a note though mentioning the flags that were in use before.

Comment 8

[Greg Mierzwinski [:sparky]]

I'm running the file 'codecoverage.py' on the one-click loaner now and everything is going well. I had to make some changes to the commands though to get it working, and I've also updated the docs with the changes.

One improvement I thought of would be to have 'codecoverage.py' output some status information. It runs idly for so long without any output that it makes you wonder if it froze or not.

Comment 9

[Marco Castelluccio [:marco]]

(In reply to Greg Mierzwinski [:gmierz] from comment #8)
> One improvement I thought of would be to have 'codecoverage.py' output some
> status information. It runs idly for so long without any output that it
> makes you wonder if it froze or not.

Could you file an issue on https://github.com/marco-c/firefox-code-coverage ?

Comment 10

[Greg Mierzwinski [:sparky]]

Will do!

Comment 11

[Marco Castelluccio [:marco]]

I think we can call this fixed, as its only dependency is fixed.