Closed Bug 1733631 Opened 2 months ago Closed 1 month ago

Don't allow variants to be keyed `by-platform`

Categories

(Firefox Build System :: Task Configuration, task)

task

Tracking

(firefox95 fixed)

RESOLVED FIXED
95 Branch
Tracking Status
firefox95 --- fixed

People

(Reporter: ahal, Assigned: ahal)

References

Details

Attachments

(30 files, 1 obsolete file)

48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review
48 bytes, text/x-phabricator-request
Details | Review

One big point of confusion that always comes up when modifying the test configs in our CI is how to get variants running in their desired locations. The issue is that there are two variables that affect this and both can be keyed by-platform: the variants and run-on-projects variables.

In order to figure out what gets run where, one must examine both of these variables, parse out the regexes and make the necessary adjustments. In this bug, I'm proposing we instead disallow keying the variants by-platform and rely solely on run-on-projects.

The benefits of this are:

  1. Only a single variable that controls where tasks are run (easier to reason about)
  2. Variants that aren't running on given platforms will still be in the full taskgraph (so developers can test them on try)
  3. Will unblock further simplifications around variants and test transforms

The cons of this are:

  1. Full taskgraph will get much larger
  2. There may be a performance impact (will measure this before landing)

IMPORTANT: This bug will not impact where tasks actually run in any way whatsoever. I will make sure I don't make any mistakes using the --diff flag.

This may help developers if they want to test running a test variant against a non-standard platform on try: they just need to add the task, rather than modify the taskgraph configs.

Assignee: nobody → ahal
Status: NEW → ASSIGNED

This does not add or remove any tasks.

This does not add or remove any tasks.

Depends on D128798

This does not add or remove any tasks.

Depends on D128799

This does not add or remove any tasks.

Depends on D128800

This does not add or remove any tasks.

Depends on D128801

This does not add or remove any tasks.

Depends on D128802

This does make the config file a bit more verbose, but makes it much easier to
understand what is running where, which seems like a good trade-off here.

Depends on D128803

Keywords: leave-open

(In reply to Andrew Halberstadt [:ahal] from comment #0)

The cons of this are:

  1. Full taskgraph will get much larger
  2. There may be a performance impact (will measure this before landing)

After doing this for mochitest.yml it adds about 2500 tasks and ~7 seconds to the generation on my machine (I expect other suite files will have much less of an impact).

However:

  1. The configs are so much easier to read
  2. I have plans to re-implement platform restrictions later (e.g, the wayland variant should only apply to Linux) in a much nicer way, so this will cut the tasks back down a bunch.
  3. This is a necessary pre-requisite to many other cleanups in the test configs.

So for those reasons I'm planning to proceed here.

Attachment #9246532 - Attachment is obsolete: true
Attachment #9246568 - Attachment description: WIP: Bug 1733631 - [ci] Stop keying 'mochitest-a11y' variants by-platform → Bug 1733631 - [ci] Stop keying 'mochitest-a11y' variants by-platform
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7eb79ae0c067
[ci] Stop keying 'mochitest-plain' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/87f4ddfc7699
[ci] Stop keying 'mochitest-browser-chrome' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/064b33d05210
[ci] Stop keying 'mochitest-devtools-chrome' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/b65981ff7d7f
[taskgraph] Don't override 'run-on-projects' from 'webgl-ipc' variant, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/ce3e32e23ce4
[ci] Stop keying 'mochitest-media' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/800f313baa0a
[ci] Stop keying 'mochitest-webgl*' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/ab63303e38de
[ci] Stop setting 'variants' in the 'job-defaults' section, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/fa3ba1da3e08
[ci] Stop keying 'mochitest-a11y' variants by-platform r=jmaher
https://hg.mozilla.org/integration/autoland/rev/b18637e10ff0
[ci] Stop keying 'mochitest-browser-chrome-failures' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/82473ffd6538
[ci] Stop keying 'mochitest-chrome' variants by-platform, r=jmaher
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8a32a43ded58
[ci] Stop keying 'mochitest-plain-gpu' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/5c555536c013
[ci] Stop keying 'mochitest-chrome-gpu' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/fffdbd2e4b88
[ci] Stop keying 'mochitest-media-failures' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/53cae44591df
[ci] Stop keying 'mochitest-plain-headless' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/1e027ab8f02e
[ci] Stop keying 'mochitest-webgpu' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/8a23bd78c0c6
[ci] Stop keying 'mochitest-remote' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/46c9fb7bd58a
[ci] Stop keying 'mochitest-browser-screenshots' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/af9935601af3
[ci] Stop setting 'variants' in defaults section of reftest.yml, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/09109ec93bfc
[ci] Stop keying 'crashtest' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/ef27b70177bc
[ci] Stop keying 'reftest' variants by-platform, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/052cfe767692
[ci] Stop keying remaining reftest suite variants by-platform, r=jmaher
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f416433d300d
[ci] Stop setting 'variants' in defaults section of web-platform.yml, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/10d407411179
[ci] Stop keying variants by-platform in 'web-platform.yml', r=gbrown
https://hg.mozilla.org/integration/autoland/rev/65364a9f6b0d
[ci] Stop using 'by-variant' in WPT tasks with no variants, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/12db8ba74c30
[ci] Stop keying variants by-platform in 'xpcshell.yml', r=gbrown
https://hg.mozilla.org/integration/autoland/rev/3f3f989e85bc
[ci] Stop setting 'variants' / 'run-on-projects' in defaults section of talos.yml, r=gbrown
https://hg.mozilla.org/integration/autoland/rev/395d570e3e0d
[ci] Stop keying variants by-platform in 'talos.yml', r=gbrown
Blocks: 1737110
Attachment #9247107 - Attachment description: Bug 1733631 - [ci] Stop keying variants by-platform in 'raptor.yml', r?gbrown → Bug 1733631 - [ci] Stop keying variants in raptor.yml and browsertime-desktop.yml, r?gbrown

This should be resolved after next batch of patches land.

Keywords: leave-open
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b6adebd58daf
[ci] Stop keying variants by-platform in 'marionette.yml', r=gbrown
https://hg.mozilla.org/integration/autoland/rev/94d832b6e884
[ci] Stop keying variants in raptor.yml and browsertime-desktop.yml, r=gbrown,perftest-reviewers,AlexandruIonescu
https://hg.mozilla.org/integration/autoland/rev/93581e1fe476
[taskgraph] Disallow keying 'variants' by anything, r=gbrown
Status: ASSIGNED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
Target Milestone: --- → 95 Branch
You need to log in before you can comment on or make changes to this bug.