Closed Bug 1629642 Opened 4 months ago Closed 4 months ago

Implement a smarter platform filter for the bugbug scheduling algorithm

Categories

(Firefox Build System :: Task Configuration, enhancement)

enhancement

Tracking

(firefox77 fixed)

RESOLVED FIXED
mozilla77
Tracking Status
firefox77 --- fixed

People

(Reporter: ahal, Assigned: ahal)

References

(Blocks 1 open bug)

Details

Attachments

(6 files)

In bug 1624902 I implemented a filter that reduces bugbug's chosen platforms to debug only. However we can do something smarter. My current idea is something like this (though may morph over time):

  1. Break each manifest into tiers based on their confidence intervals, e.g:
    a. Tier 1: >0.9
    b. Tier 2: >0.75
    c. Tier 3: >0.5
    d. Tier 4: >0

  2. Assign constraints to each tier, e.g:

    a. A manifest in tier 1 is guaranteed to run in at least one task from every major configuration.
    b. A manifest in tier 2 is guaranteed to run in at least one task from every OS family.
    c. A manifest in tier 3 is guaranteed to run in at least one mobile and one desktop platform.
    d. A manifest in tier 4 is guaranteed to run in at least one task

  3. Overall, try to cover as many different configurations as possible while satisfying the above constraints. This way we try to spread out our coverage across as many configurations as possible while minimizing duplication.

This is just an example, we may end up with more or fewer tiers, different constraints, different thresholds etc.

This change is beneficial for two reasons:

  1. Improvement on the single responsibility principle
  2. Platform filter isn't 'bugbug' specific. E.g, the 'relevant tests' optimizer
    could also theoretically use this. Having it as a standalone optimizer allows
    us to compose it with other strategies.

Hm, implementing the above as stated won't be possible. We won't be able to make those guarantees as the optimizers process tasks one at a time, so have no idea what is coming. I'm implementing something a lot simpler for now, and we can iterate on it over time.

Keywords: leave-open
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f94d27cca400
[taskgraph] Pull bugbug's 'platform filter' into a standalone strategy, r=marco
Attachment #9141236 - Attachment description: Bug 1629642 - [taskgraph] Weight tasks that contain multiple groups higher in bugbug strategy, r?marco → Bug 1629642 - [taskgraph] Add an option to sum confidence thresholds of manifests in bugbug scheduler, r?marco
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/25119864bcc1
[taskgraph] Add an option to sum confidence thresholds of manifests in bugbug scheduler, r=marco
Attachment #9143708 - Attachment description: Bug 1629642 - [ci] Fix wrong strategy name in bugbug_combined_high shadow-scheduler, r?marco → Bug 1629642 - [ci] Fix wrong strategy name in bugbug_combined_high shadow-scheduler, DONTBUILD, r?marco
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d7d94860086b
[ci] Fix wrong strategy name in bugbug_combined_high shadow-scheduler, DONTBUILD, r=marco
Keywords: leave-open
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0414f57713e8
[taskgraph] Split 'test' optimization strategy arg into two r=marco
https://hg.mozilla.org/integration/autoland/rev/c5df25796d9b
[taskgraph] Implement a 'disperse' platform optimization strategy r=marco
https://hg.mozilla.org/integration/autoland/rev/a34695d9b99d
[ci] Add a shadow-scheduler for the 'bugbug.disperse' scheduling algorithm, r=marco
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla77
You need to log in before you can comment on or make changes to this bug.