1427312 - Add mechanism to create tasks to compare firefox builds

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Mike Hommey [:glandium]]

I've been comparing firefox builds a *lot* in 3 distinct occasions, and other people have done that too in the past (iirc Callek and maybe pmoore). This time, I've done a lot more comparisons than I have in the past, so I went the extra mile and automated it. I think it would be useful to land the facility.

Plus I'm going to attach non-land-worthy patches relying on it to bugs leading to bug 1399679)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

There are e.g. some build infrastructure changes that we want to have a
controlled impact on the Firefox builds we produce. We have, in multiple
occasions, gone through manual work to compare Firefox builds, most of
the time using the diffoscope tool (https://diffoscope.org/).

This change introduces a new task kind that takes two Firefox builds as
input, either by name (reference to a build from the current task graph)
or by index (reference to a build from a previous push), and compares
them.

In order to get a Firefox build by index, we rely on dummy tasks with
an optimization we expect to always hit, so we add the necessary bits
to ensure those dummy tasks can go through up to the optimization phase
and be optimized out there.

Review commit: https://reviewboard.mozilla.org/r/209468/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/209468/

Comment 2

[Mike Hommey [:glandium]]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209468/diff/1-2/

Comment 3

[Mike Hommey [:glandium]]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209468/diff/2-3/

Comment 4

[Mike Hommey [:glandium]]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209468/diff/3-4/

Comment 5

[Mike Hommey [:glandium]]

Examples of use:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=52774e108c429fdad181a8812c151dcf4f2051d5
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c8fa5b65913b5290c01fe083be58b586543d11e2 (see second patch in bug 1427316)

Comment 6

[Mike Hommey [:glandium]]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209468/diff/4-5/

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

https://reviewboard.mozilla.org/r/209468/#review215462

Nicely done -- especially the final verification.

::: taskcluster/docs/kinds.rst:358
(Diff revision 5)
>  ------------------------------
>  Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task.
> +
> +diffoscope
> +----------
> +Tasks used to compare pairs of Firefox builds using https://diffoscope.org/.

From what I can tell, the intent is to do this in a try push?  A bit of extra instruction here might help someone figure that out later..

Comment 8

[Nick Alexander :nalexander [he/him]]

(In reply to Dustin J. Mitchell [:dustin] from comment #7)
> Comment on attachment 8939044 [details]
> Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.
> 
> https://reviewboard.mozilla.org/r/209468/#review215462
> 
> Nicely done -- especially the final verification.
> 
> ::: taskcluster/docs/kinds.rst:358
> (Diff revision 5)
> >  ------------------------------
> >  Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task.
> > +
> > +diffoscope
> > +----------
> > +Tasks used to compare pairs of Firefox builds using https://diffoscope.org/.
> 
> From what I can tell, the intent is to do this in a try push?  A bit of
> extra instruction here might help someone figure that out later..

Yeah -- more deets needed on how to use this.  Looks like the examplar use in the binutils ticket (https://bugzilla.mozilla.org/show_bug.cgi?id=1427316) predates this work, so it's not good as a model.

Comment 9

[Mike Hommey [:glandium]]

(In reply to Nick Alexander :nalexander (less responsive until Jan 3, 2018) from comment #8)
> Yeah -- more deets needed on how to use this.  Looks like the examplar use
> in the binutils ticket
> (https://bugzilla.mozilla.org/show_bug.cgi?id=1427316) predates this work,
> so it's not good as a model.

It doesn't predate this, fwiw. It uses it.

Comment 10

[Mike Hommey [:glandium]]

Comment on attachment 8939044 [details]
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/209468/diff/5-6/

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a4b21febc58d
Add mechanism to create tasks to compare Firefox builds. r=dustin

Comment 12

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/a4b21febc58d