don't run shippable builds or tests on try in most cases
Categories
(Testing :: General, enhancement)
Tracking
(firefox78 fixed)
Tracking | Status | |
---|---|---|
firefox78 | --- | fixed |
People
(Reporter: bhearsum, Assigned: bhearsum, NeedInfo)
References
(Blocks 1 open bug)
Details
(Whiteboard: [ci-costs-2020:done])
Attachments
(1 file)
opt & shippable builds are very much alike in build configuration, and there aren't many cases where we should be running both of them on try. Opt builds are much, much faster, so they should be the preferred build for running mochitest/reftest/etc. on. Marco did some investigation and found very few cases where testing shippable builds found unique failures compared to opt.
We do still need the ability to run shippable builds & tests on try for certain types of work. I propose that shippable builds & tests should not be scheduled unless:
a) performance tests are requested (we don't do these for opt)
or
b) shippable builds are explicitly requested
This means that if, eg "mochitest" is selected, shippable builds & tests will not be run unless an explicit action is taken.
Assignee | ||
Updated•5 years ago
|
Updated•5 years ago
|
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 1•5 years ago
|
||
glandium, I started digging into this and it looks like we can probably just add an entry to https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/target_tasks.py#21 that matches all of the shippable tasks that we also have opt tasks for. Does that sound right to you?
Comment 2•5 years ago
|
||
I'm not entirely sure. That might prevent the talos and raptor tasks from being in the target set.
Assignee | ||
Comment 3•5 years ago
|
||
Comment 5•5 years ago
|
||
bugherder |
Assignee | ||
Comment 6•5 years ago
|
||
I've been using https://datastudio.google.com/reporting/1QG2CTQV1KCu0Vavdr0HZZNzKF5ykxocn/page/kxCQB/edit and https://console.cloud.google.com/bigquery?project=moz-fx-data-taskclu-prod-8fbf&supportedpurview=project&j=bq:US:bquxjob_675116de_17257dc13d5&page=queryresults to help calculate savings here.
Updated•5 years ago
|
Description
•