Code coverage test jobs should set GCOV_PREFIX, package .gcda files and upload them to blobber

RESOLVED FIXED

Status

Release Engineering
Mozharness
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments, 3 obsolete attachments)

See https://bugzilla.mozilla.org/show_bug.cgi?id=890116#c1

Code coverage builds and tests are running on Cedar. The builds are also uploading the .gcno zip file. The last piece is to get the test jobs to upload the .gcda files.
Depends on: 1079924
Created attachment 8501794 [details] [diff] [review]
Schedule linux64-cc on Ash
Attachment #8501794 - Flags: review?(jlund)
Comment on attachment 8501794 [details] [diff] [review]
Schedule linux64-cc on Ash

Ugh, sorry.. attached this to the wrong bug.
Attachment #8501794 - Attachment is obsolete: true
Attachment #8501794 - Flags: review?(jlund)
Blocks: 1110465
Created attachment 8555349 [details] [diff] [review]
Code coverage mixin

This is a mixin I played with on code coverage tests. Since this is my push-to-try stuff, I've not concerned myself with trying to hide it behind a flag, but this seems to give results for Linux64 fully (including the web-platform-tests, which I haven't gotten before).

Ignore the code-coverage-gcno.zip stuff--I was trying to get Android builds to not think it the installer before I gave up and just renamed the file output.
Created attachment 8555438 [details] [diff] [review]
Another (very similar) code coverage mixin

Heh, I actually had this very similar patch hanging around locally for awhile. I do plan to get this landed, but it keeps getting bumped by my quarterly goals. With luck, I'll have some time to work on this this quarter.

In the meantime I don't want to block other people from working on it if they want, so temporarily unassigning myself.
Assignee: ahalberstadt → nobody
Status: ASSIGNED → NEW
Going to pick this up again. Sorry for the long delay.
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
Attachment #8555438 - Attachment is obsolete: true
Created attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

/r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package .gcda files and upload them to blobber, r=jlund

Pull down this commit:

hg pull -r 9fda257cf65774e319a14a5bc438a593d157aedf https://reviewboard-hg.mozilla.org/build-mozharness
Attachment #8598289 - Flags: review?(jlund)
Sorry this adds a new mixin.. if you'd like me to implement it another way, feel free to give me an r- and let me know what you'd prefer!

Comment 8

2 years ago
(In reply to Andrew Halberstadt [:ahal] from comment #6)
> Created attachment 8598289 [details]
> MozReview Request: bz://1059951/ahal
> 
> /r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package
> .gcda files and upload them to blobber, r=jlund
> 
> Pull down this commit:
> 
> hg pull -r 9fda257cf65774e319a14a5bc438a593d157aedf
> https://reviewboard-hg.mozilla.org/build-mozharness

sorry ahal, I'm actually on PTO now until May 13th. Could I 302 this to someone else? Possibly someone from ateam or mgerva/catlee in releng

Updated

2 years ago
Attachment #8598289 - Flags: review?(jlund)

Comment 9

2 years ago
Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://reviewboard.mozilla.org/r/7721/#review6531

apologies, I'm on PTO until May 13th.
Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

/r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package .gcda files and upload them to blobber, r=jgriffin

Pull down this commit:

hg pull -r 6425568fbf5397575f4529caeee4058e413361f3 https://reviewboard-hg.mozilla.org/build-mozharness
Attachment #8598289 - Flags: review?(jgriffin)
jlund: No worries, have a good break!

jgriffin: this adds the ability to generate .gcda files, I'll need to make a buildbot-configs change to pass in --code-coverage for those jobs next.
For those curious, here is a try run with everything working together:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4b39871a9eba

Grab the 'code-coverage-gcno.zip' file from the build directory. Grab 'code-coverage-gcda.zip' files from the individual test jobs on blobber (click job and look under blobber urls on the bottom left). Some kind of automated tooling will be needed to download and merge the .gcda files (if desired), but I think this should be done by external process.

I know how to fix the red browser-chrome jobs. Not sure how to fix the make check failure, but it doesn't seem to be hurting anything, so I'll ignore it for now.
Attachment #8598289 - Flags: review?(jgriffin) → review+
Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://reviewboard.mozilla.org/r/7721/#review6571

lgtm

::: mozharness/mozilla/testing/codecoverage.py:50
(Diff revision 2)
> +        for root, dirs, files in os.walk(self.gcov_dir):

I guess this is necessary because the directory structure in self.gcov_dir is not consistent among platforms or jobs?
I'm not entirely sure how consistent the directory structure is tbh. Whatever generates the .gcda files puts the entire absolute path into self.gcov_dir, e.g:

/tmp/tmp123abc/build/slave/try-l6-0000000000000000/src/dom/Foo.gcda

We just need to make sure that the gcno.zip and the gcda.zip have the same structure relative to one another, so making the zipfiles start at the topsrcdir seems easiest. If the directory depth is always constant, I could do something like:

d = '/'.join(d.split('/')[5:])

But that seems even more fragile than the os.walk approach.
Could we just look for the first "src" dir?
Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://hg.mozilla.org/build/mozharness/rev/bf20a548b325

On irc we decided that pretty much any method of finding the topsrcdir would be fragile, so decided to leave things as is for now. Code coverage is still experimental so we can make adjustments later if this method doesn't work out.
Attachment #8598289 - Flags: checked-in+
This got merged to production and the in-tree pointer was bumped.
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal
Attachment #8598289 - Attachment is obsolete: true
Attachment #8618298 - Flags: review+
Created attachment 8618298 [details]
MozReview Request: Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package .gcda files and upload them to blobber, r=jgriffin
You need to log in before you can comment on or make changes to this bug.