Closed Bug 1497860 Opened 7 years ago Closed 7 years ago

Declare mozconfig in build task definition

Categories

(Firefox Build System :: Task Configuration, task)

Product:
Firefox Build System
Component:
Task Configuration
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(firefox-esr60 fixed, firefox64 fixed, firefox65 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla65
Tracking Flags:
Tracking Status
firefox-esr60 --- fixed
firefox64 --- fixed
firefox65 --- fixed

People

(Reporter: ted, Assigned: tomprince)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files)

Bug 1497860: [taskgraph] Use yaml lists for mozharness actions; r?ted
46 bytes, text/x-phabricator-request
Details | Review
Bug 1497860: Move mozconfig variant definitions to taskcluster config.
46 bytes, text/x-phabricator-request
Details | Review
Bug 1497860: [taskgraph] Ensure that mozharness actions don't contain spaces; r?Callek
46 bytes, text/x-phabricator-request
Details | Review
As an incremental step towards bug 1458907, we should have the build task definitions in the YAML files declare what mozconfig they use. Adding a new build variant is fairly easy these days, since you can just edit the in-tree task definitions. However, the most annoying boilerplate part of it is having to copy a mozharness subconfig just so you can point at a different mozconfig. For example, the linux64/debug build is defined here: https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#164-195 It specifies all of: ``` run: using: mozharness actions: [get-secrets build check-test] config: - builds/releng_base_firefox.py - builds/releng_base_linux_64_builds.py script: "mozharness/scripts/fx_desktop_build.py" secrets: true custom-build-variant-cfg: debug ``` And then we have to have this mozharness sub-config: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/builds/releng_sub_linux_configs/64_debug.py which has `'mozconfig_variant': 'debug'`. Ideally we'd just have: ``` run: using: mozharness actions: [get-secrets build check-test] mozconfig: browser/config/mozconfigs/linux64/debug ``` That part should be easy enough to change in the build transforms somewhere, just take that `mozconfig` key and set it as `MOZCONFIG` in the environment. We will probably also need to fix the `_get_mozconfig` logic in the mozharness script to handle this: https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/testing/mozharness/mozharness/mozilla/building/buildbase.py#925-943 It might take a little more work to remove the subconfigs entirely, I'll file a separate bug on that.
Blocks: 1497862
(In reply to Mike Hommey [:glandium] from comment #1) > mozconfig_variant can already be set in the taskcluster config: > https://dxr.mozilla.org/mozilla-central/rev/ > c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#671 Note that anything that can appear in mozharness subconfigs can probably appear in extra-config there.
(In reply to Mike Hommey [:glandium] from comment #1) > mozconfig_variant can already be set in the taskcluster config: > https://dxr.mozilla.org/mozilla-central/rev/ > c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#671 Interesting. I'd like to just get rid of all the mozharness logic for locating mozconfigs and have topsrcdir-relative paths directly in the task definitions. I don't think the extra layer of indirection in mozharness buys us anything these days.
> just take that `mozconfig` key and set it as `MOZCONFIG` in the environment Why not just set MOZCONFIG in the environment in the task definition?
(In reply to Mike Hommey [:glandium] from comment #4) > > just take that `mozconfig` key and set it as `MOZCONFIG` in the environment > > Why not just set MOZCONFIG in the environment in the task definition? On possible reason is that some builds (nightly in particular, but also others (which may be accidental)) get a different mozconfig depending on the branch set in `branch_specifics.py`. Adding the variation in that taskgraph configuration is slightly easier if there is specific key for it.
(In reply to Tom Prince [:tomprince] from comment #5) > On possible reason is that some builds (nightly in particular, but also > others (which may be accidental)) get a different mozconfig depending on the > branch set in `branch_specifics.py`. Adding the variation in that taskgraph > configuration is slightly easier if there is specific key for it. I didn't have a very strong argument here but having a dedicated key does make it a bit easier to work with in the taskgraph code, and we could presumably enforce a schema on build task definitions at some point.
Assignee: nobody → mozilla
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/ebd900741779 Move mozconfig variant definitions to taskcluster config. r=ted
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/3daa70ac76f6 Move mozconfig variant definitions to taskcluster config. r=ted
Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/da78ac0644ac [taskgraph] Fix typo in static-analysis-autotest config; r=tomprince
Flags: needinfo?(mozilla)
https://hg.mozilla.org/mozilla-central/rev/3daa70ac76f6 https://hg.mozilla.org/mozilla-central/rev/da78ac0644ac
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox65: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/c5bded90844e [taskgraph] Use yaml lists for mozharness actions; r=Callek https://hg.mozilla.org/integration/autoland/rev/306bbd591645 [taskgraph] Ensure that mozharness actions don't contain spaces; r=Callek
Pushed by mozilla@jorgk.com: https://hg.mozilla.org/comm-central/rev/531dfcb2d29d Port bug 1497860 - add commas to actions. rs-bustage-fix
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: