Closed Bug 1497860 Opened Last year Closed Last year

Declare mozconfig in build task definition

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(firefox-esr60 fixed, firefox64 fixed, firefox65 fixed)

RESOLVED FIXED
mozilla65
Tracking Status
firefox-esr60 --- fixed
firefox64 --- fixed
firefox65 --- fixed

People

(Reporter: ted, Assigned: tomprince)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files)

As an incremental step towards bug 1458907, we should have the build task definitions in the YAML files declare what mozconfig they use. Adding a new build variant is fairly easy these days, since you can just edit the in-tree task definitions. However, the most annoying boilerplate part of it is having to copy a mozharness subconfig just so you can point at a different mozconfig.

For example, the linux64/debug build is defined here:
https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#164-195

It specifies all of:

```
    run:
        using: mozharness
        actions: [get-secrets build check-test]
        config:
            - builds/releng_base_firefox.py
            - builds/releng_base_linux_64_builds.py
        script: "mozharness/scripts/fx_desktop_build.py"
        secrets: true
        custom-build-variant-cfg: debug
```

And then we have to have this mozharness sub-config:
https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/builds/releng_sub_linux_configs/64_debug.py

which has `'mozconfig_variant': 'debug'`.

Ideally we'd just have:
```
    run:
        using: mozharness
        actions: [get-secrets build check-test]
        mozconfig: browser/config/mozconfigs/linux64/debug
```

That part should be easy enough to change in the build transforms somewhere, just take that `mozconfig` key and set it as `MOZCONFIG` in the environment. We will probably also need to fix the `_get_mozconfig` logic in the mozharness script to handle this:
https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/testing/mozharness/mozharness/mozilla/building/buildbase.py#925-943

It might take a little more work to remove the subconfigs entirely, I'll file a separate bug on that.
(In reply to Mike Hommey [:glandium] from comment #1)
> mozconfig_variant can already be set in the taskcluster config: 
> https://dxr.mozilla.org/mozilla-central/rev/
> c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#671

Note that anything that can appear in mozharness subconfigs can probably appear in extra-config there.
(In reply to Mike Hommey [:glandium] from comment #1)
> mozconfig_variant can already be set in the taskcluster config: 
> https://dxr.mozilla.org/mozilla-central/rev/
> c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/build/linux.yml#671

Interesting. I'd like to just get rid of all the mozharness logic for locating mozconfigs and have topsrcdir-relative paths directly in the task definitions. I don't think the extra layer of indirection in mozharness buys us anything these days.
> just take that `mozconfig` key and set it as `MOZCONFIG` in the environment

Why not just set MOZCONFIG in the environment in the task definition?
(In reply to Mike Hommey [:glandium] from comment #4)
> > just take that `mozconfig` key and set it as `MOZCONFIG` in the environment
> 
> Why not just set MOZCONFIG in the environment in the task definition?

On possible reason is that some builds (nightly in particular, but also others (which may be accidental)) get a different mozconfig depending on the branch set in `branch_specifics.py`. Adding the variation in that taskgraph configuration is slightly easier if there is specific key for it.
(In reply to Tom Prince [:tomprince] from comment #5)
> On possible reason is that some builds (nightly in particular, but also
> others (which may be accidental)) get a different mozconfig depending on the
> branch set in `branch_specifics.py`. Adding the variation in that taskgraph
> configuration is slightly easier if there is specific key for it.

I didn't have a very strong argument here but having a dedicated key does make it a bit easier to work with in the taskgraph code, and we could presumably enforce a schema on build task definitions at some point.
Assignee: nobody → mozilla
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/ebd900741779
Move mozconfig variant definitions to taskcluster config. r=ted
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/3daa70ac76f6
Move mozconfig variant definitions to taskcluster config. r=ted
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/da78ac0644ac
[taskgraph] Fix typo in static-analysis-autotest config; r=tomprince
Flags: needinfo?(mozilla)
https://hg.mozilla.org/mozilla-central/rev/3daa70ac76f6
https://hg.mozilla.org/mozilla-central/rev/da78ac0644ac
Status: NEW → RESOLVED
Closed: Last year
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/c5bded90844e
[taskgraph] Use yaml lists for mozharness actions; r=Callek
https://hg.mozilla.org/integration/autoland/rev/306bbd591645
[taskgraph] Ensure that mozharness actions don't contain spaces; r=Callek
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/531dfcb2d29d
Port bug 1497860 - add commas to actions. rs-bustage-fix
You need to log in before you can comment on or make changes to this bug.