Closed Bug 1334402 Opened 7 years ago Closed 7 years ago

gcno archives are incomplete - bad interaction with sccache

Categories

(Testing :: Code Coverage, defect)

defect
Not set
normal

Tracking

(firefox54 fixed)

RESOLVED FIXED
mozilla54
Tracking Status
firefox54 --- fixed

People

(Reporter: u587052, Unassigned)

References

Details

Attachments

(1 file, 2 obsolete files)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161213204721

Steps to reproduce:

1. Run a build on try: |try: -b o -p linux64-ccov -u none -t none|
2. Wait for build to finish


Actual results:

Once the build completes, the Linux x64 ccov task "target.code-coverage-gcno.zip" artifact (See Job Details) is missing many gcno files. Re-running the build again creates an incomplete "gcno" archive, but it is missing different gcno files


Expected results:

All gcno files produced should exist in the archive: it should be around 75.4mB
Attachment #8831059 - Flags: review?(gmierz2)
Build running on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7091c33301d6f3fc62f3cb3fc67a5ed20d8dd5f0&selectedJob=72485810

If I understood the taskcluster config correctly, the gcno file archive should be 75.4mB on the Linux x64 ccov task.
Note: changes made according to direction from https://bugzilla.mozilla.org/show_bug.cgi?id=1332917#c22
Summary: gcno/gcda archives are incomplete - bad interaction with sccache → gcno archives are incomplete - bad interaction with sccache
Note #2: the gcno archive created in the above try is too small. sccache was still used in that build, probably
Comment on attachment 8831059 [details]
Bug 1334402 - Don't use sccache in taskcluster builds +560562

https://reviewboard.mozilla.org/r/107724/#review108970

Good progress so far Mitch!

::: taskcluster/taskgraph/transforms/build.py:22
(Diff revision 1)
>  def set_defaults(config, jobs):
>      """Set defaults, including those that differ per worker implementation"""
>      for job in jobs:
>          job['treeherder'].setdefault('kind', 'build')
>          job['treeherder'].setdefault('tier', 1)
> -        job.setdefault('needs-sccache', True)
> +        job.setdefault('needs-sccache', False)

This change will cause all jobs - even those outside of the linux64-ccov build - to not use the sccache. This is because it's a transform that sets the defaults for all and any build tasks, see https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/build.py#4-7.

The task schema for builds lives here: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/task.py#135

And we would need to set it in the file: https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/build/linux.yml#325 

That should hopefully give us the consistent file archive we are looking for.
Attachment #8831059 - Flags: review?(gmierz2) → review-
Oh, sweet, so those python files define the schema, which is then configured for each type of build in a nifty ".yml". Makes sense, I'll make the change strictly to the linux64-ccov build.
Blocks: 1334940
Attachment #8831578 - Flags: review?(gmierz2)
Attachment #8831579 - Flags: review?(gmierz2)
This try build has the changes, and is successfully producing a full gcno archive: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f0fef5a60cad6f6c402bac3d9ee1af1cd1562c42
Attachment #8831578 - Attachment is obsolete: true
Attachment #8831578 - Flags: review?(gmierz2)
Attachment #8831579 - Attachment is obsolete: true
Attachment #8831579 - Flags: review?(gmierz2)
Comment on attachment 8831059 [details]
Bug 1334402 - Don't use sccache in taskcluster builds +560562

https://reviewboard.mozilla.org/r/107724/#review109358

Nice work on fixing this bug!
Attachment #8831059 - Flags: review?(gmierz2) → review+
Joel, can we land these changes?

They seem good to me and it also fixes more than just the problem we are facing with gcno archives - the build definition flag 'no-sccache' was not being handled properly. And since this build is the first to not use the sccache, it is the first to have to deal with it. There's a taskcluster job in comment #9 that shows that it is working well.
Flags: needinfo?(jmaher)
Comment on attachment 8831059 [details]
Bug 1334402 - Don't use sccache in taskcluster builds +560562

https://reviewboard.mozilla.org/r/107722/#review109380

looks good
Comment on attachment 8831059 [details]
Bug 1334402 - Don't use sccache in taskcluster builds +560562

https://reviewboard.mozilla.org/r/107724/#review109382
Attachment #8831059 - Flags: review+
yes, this will go in shortly
Flags: needinfo?(jmaher)
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1def3145c82a
Don't use sccache in taskcluster builds r=gmierz,jmaher+560562
https://hg.mozilla.org/mozilla-central/rev/1def3145c82a
Status: UNCONFIRMED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: