Closed Bug 1291473 Opened 8 years ago Closed 8 years ago

build task cleanup

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: dustin, Assigned: dustin)

References

Details

Attachments

(11 files, 1 obsolete file)

Bug 1291473: set tier for st-an builds;
58 bytes, text/x-review-board-request
Callek
: review+
Details
Bug 1291473: set tiers explicitly;
58 bytes, text/x-review-board-request
mshal
: review+
Details
Bug 1291473: set TOOLTOOL_CACHE for android-api-15* builds;
58 bytes, text/x-review-board-request
gbrown
: review+
Details
Bug 1291473: set mulet tier explicitly
58 bytes, text/x-review-board-request
gerard-majax
: review+
Details
Bug 1291473: set tier explicitly for android partner builds;
58 bytes, text/x-review-board-request
jlund
: review+
Details
Bug 1291473: mark valgrind builds explicitly as tier-1;
58 bytes, text/x-review-board-request
kmoir
: review+
Details
Bug 1291473: set tier explicitly for SM jobs;
58 bytes, text/x-review-board-request
sfink
: review+
Details
Bug 1291473: move hazard tooltool cache to the usual location;
58 bytes, text/x-review-board-request
sfink
: review+
Details
Bug 1291473: set tier explicitly for SM and Hazard jobs;
58 bytes, text/x-review-board-request
sfink
: review+
Details
Bug 1291473: accept pushdate from command line;
58 bytes, text/x-review-board-request
garndt
: review+
Details
Bug 1291473: put l10n artifacts in the same place as other builds;
58 bytes, text/x-review-board-request
Callek
: review+
Details 
This is a collection of minor, hopefully impact-free tweaks to existing build tasks to make them a little more consistent.  It comes from my work on bug 1286075 generating build tasks using transforms.

I make no promises that these are all the tweaks I will make, but it's the bulk of them!
Review commit: https://reviewboard.mozilla.org/r/68720/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/68720/
Attachment #8777121 - Flags: review?(ehsan)
Attachment #8777122 - Flags: review?(mshal)
Attachment #8777123 - Flags: review?(gbrown)
Attachment #8777125 - Flags: review?(jlund)
Attachment #8777126 - Flags: review?(kmoir)
Attachment #8777127 - Flags: review?(sphink)
Attachment #8777128 - Flags: review?(sphink)
Attachment #8777129 - Flags: review?(sphink)
Attachment #8777131 - Flags: review?(garndt)
Attachment #8777132 - Flags: review?(bugspam.Callek)
I'm not sure who to ask about removing device builds.
Flags: needinfo?(catlee)
Attachment #8777124 - Flags: review?(lissyx+mozillians)
Comment on attachment 8777123 [details]
Bug 1291473: set TOOLTOOL_CACHE for android-api-15* builds;

https://reviewboard.mozilla.org/r/68724/#review65752
Attachment #8777123 - Flags: review?(gbrown) → review+
Comment on attachment 8777127 [details]
Bug 1291473: set tier explicitly for SM jobs;

https://reviewboard.mozilla.org/r/68732/#review65772
Attachment #8777127 - Flags: review?(sphink) → review+
Comment on attachment 8777128 [details]
Bug 1291473: move hazard tooltool cache to the usual location;

https://reviewboard.mozilla.org/r/68734/#review65774
Attachment #8777128 - Flags: review?(sphink) → review+
Comment on attachment 8777129 [details]
Bug 1291473: set tier explicitly for SM and Hazard jobs;

https://reviewboard.mozilla.org/r/68736/#review65776
Attachment #8777129 - Flags: review?(sphink) → review+
Comment on attachment 8777124 [details]
Bug 1291473: set mulet tier explicitly

https://reviewboard.mozilla.org/r/68726/#review65920
Attachment #8777124 - Flags: review?(lissyx+mozillians) → review+
Comment on attachment 8777126 [details]
Bug 1291473: mark valgrind builds explicitly as tier-1;

https://reviewboard.mozilla.org/r/68730/#review65970
Attachment #8777126 - Flags: review?(kmoir) → review+
Comment on attachment 8777131 [details]
Bug 1291473: accept pushdate from command line;

https://reviewboard.mozilla.org/r/68740/#review65984

::: .taskcluster.yml:78
(Diff revision 1)
>            # Arguments passed into `mach taskgraph decision`
>            # TODO use mozilla-unified for the base repository once the tc-vcs
>            # tar.gz archives are created or tc-vcs isn't being used.
>            DECISION_ARGS: >
>              --pushlog-id='{{pushlog_id}}'
> +            --pushdate='{{pushdate}}'

This will be dependent on the PR that you opened for mozilla-taskcluster (left a comment on that PR) so we'll just need to land that before this.
Attachment #8777131 - Flags: review?(garndt) → review+
Comment on attachment 8777122 [details]
Bug 1291473: set tiers explicitly;

Looks good! I didn't look through all the patches in this series - are we going to eventually disallow omitting the tier instead of defaulting it to 1?
Attachment #8777122 - Flags: review?(mshal) → review+
Yes, all taskcluster tasks will have tiers specified so there's no ambiguity.
Comment on attachment 8777121 [details]
Bug 1291473: set tier for st-an builds;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68720/diff/1-2/
Attachment #8777124 - Attachment description: Bug 1291473: set mulet tier explicitly → Bug 1291473: set mulet tier explicitly?
Attachment #8777122 - Flags: review+ → review?(mshal)
Comment on attachment 8777122 [details]
Bug 1291473: set tiers explicitly;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68722/diff/1-2/
Comment on attachment 8777123 [details]
Bug 1291473: set TOOLTOOL_CACHE for android-api-15* builds;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68724/diff/1-2/
Comment on attachment 8777124 [details]
Bug 1291473: set mulet tier explicitly

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68726/diff/1-2/
Comment on attachment 8777125 [details]
Bug 1291473: set tier explicitly for android partner builds;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68728/diff/1-2/
Comment on attachment 8777126 [details]
Bug 1291473: mark valgrind builds explicitly as tier-1;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68730/diff/1-2/
Comment on attachment 8777127 [details]
Bug 1291473: set tier explicitly for SM jobs;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68732/diff/1-2/
Comment on attachment 8777128 [details]
Bug 1291473: move hazard tooltool cache to the usual location;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68734/diff/1-2/
Comment on attachment 8777129 [details]
Bug 1291473: set tier explicitly for SM and Hazard jobs;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68736/diff/1-2/
Comment on attachment 8777131 [details]
Bug 1291473: accept pushdate from command line;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68740/diff/1-2/
Comment on attachment 8777132 [details]
Bug 1291473: put l10n artifacts in the same place as other builds;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/68742/diff/1-2/
Attachment #8777130 - Attachment is obsolete: true
Comment on attachment 8777125 [details]
Bug 1291473: set tier explicitly for android partner builds;

https://reviewboard.mozilla.org/r/68728/#review66416

Coop confirms tier 2 is correct.
Attachment #8777125 - Flags: review+
Comment on attachment 8777132 [details]
Bug 1291473: put l10n artifacts in the same place as other builds;

https://reviewboard.mozilla.org/r/68742/#review66430

The reason I did build/ as the artifact location was to get the mozharness logs uploaded too. But I don't think thats a blocker here. (And we can always do something like https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/tests/make_task_description.py#28 anyway
Attachment #8777132 - Flags: review?(bugspam.Callek) → review+
Attachment #8777125 - Flags: review?(jlund)
(In reply to Dustin J. Mitchell [:dustin] from comment #13)
> I'm not sure who to ask about removing device builds.

You can remove them if it's blocking your work here. r=catlee
Flags: needinfo?(catlee)
Comment on attachment 8777121 [details]
Bug 1291473: set tier for st-an builds;

https://reviewboard.mozilla.org/r/68720/#review67288

With KWierso's help I determined these are indeed tier1 on treeherder at present. And this patch just matches that, so we can clear a pending r? from ehsan and unblock you here.
Attachment #8777121 - Flags: review+
Attachment #8777121 - Flags: review?(ehsan)
Pushed by dmitchell@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/40d52c65a05b
set tier for st-an builds; r=Callek
https://hg.mozilla.org/integration/autoland/rev/db6b8d9939b0
set tiers explicitly; r=mshal
https://hg.mozilla.org/integration/autoland/rev/879fb268bbd1
set TOOLTOOL_CACHE for android-api-15* builds; r=gbrown
https://hg.mozilla.org/integration/autoland/rev/06129c0b8c89
set mulet tier explicitly r=gerard-majax
https://hg.mozilla.org/integration/autoland/rev/9e30ed5077ae
set tier explicitly for android partner builds; r=Callek
https://hg.mozilla.org/integration/autoland/rev/b04b386a989f
mark valgrind builds explicitly as tier-1; r=kmoir
https://hg.mozilla.org/integration/autoland/rev/629e1c3ff500
set tier explicitly for SM jobs; r=sfink
https://hg.mozilla.org/integration/autoland/rev/e1ab97b5c5ee
move hazard tooltool cache to the usual location; r=sfink
https://hg.mozilla.org/integration/autoland/rev/1fc133437bb0
set tier explicitly for SM and Hazard jobs; r=sfink
https://hg.mozilla.org/integration/autoland/rev/fc1ed3fd8584
accept pushdate from command line; r=garndt
https://hg.mozilla.org/integration/autoland/rev/7d0cb37025de
put l10n artifacts in the same place as other builds; r=Callek
I backed out the pushdate commit because it broke gecko-decision tasks.
Backout by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/fc6ed18f76e1
Backed out changeset fc1ed3fd8584 on suspicion of breaking decision tasks
Flags: needinfo?(dustin)
Flags: needinfo?(dustin)
So it looks like when you ask people if they still need builds and they say yes, you still remove them. Thanks.
I reverted the removal of the device builds -- they will stay in the legacy kind with a few other builds that I can't see how best to forward-port to the new system, and I will file bugs to port them after bug 1286075 lands.
(In reply to Dustin J. Mitchell [:dustin] from comment #61)
> I reverted the removal of the device builds -- they will stay in the legacy
> kind with a few other builds that I can't see how best to forward-port to
> the new system, and I will file bugs to port them after bug 1286075 lands.

Thanks! I had a look at your branch on that bug 1286075, and I am curious to know what would be blocking moving these device builds to the new scheme? As much as I could see, it seems pretty straightforward, so I guess that there is a devil in some details that I missed.
Blocks: 1284602
If I've done my job right there, it won't be too difficult, but let's see what things look like when all of the dust has settled.
Comment on attachment 8777125 [details]
Bug 1291473: set tier explicitly for android partner builds;

r+ via callek but mozreview confusion

2:42:00 <Callek> dustin: did you mean to re-flag jlund on Bug 1291473
12:42:02 <firebot> https://bugzil.la/1291473 — NEW, dustin@mozilla.com — build task cleanup
12:42:22 <dustin> no, no idea why it did that
Attachment #8777125 - Flags: review?(jlund) → review+
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: