Closed Bug 1734141 Opened 3 years ago Closed 2 years ago

Try estimate for jobs is unclear - "estimates: In the 79% percentile" -- is my push in the lowest or highest 79%?

Categories

(Developer Infrastructure :: Try, defect, P3)

Desktop
All

Tracking

(firefox97 fixed)

RESOLVED FIXED
97 Branch
Tracking Status
firefox97 --- fixed

People

(Reporter: Gijs, Assigned: jgilbert)

References

Details

Attachments

(1 file)

We could do with saying something like "24% of trypushes are faster/slower than yours".

Severity: -- → S4
Priority: -- → P3

Avoid "percentile" for user output, since it's not clear if e.g. a 10th
percentile push duration is good or bad.
(Also the previous output incorrectly displayed "10% percentile" for a
90th percentile push duration estimate)

Assignee: nobody → jgilbert
Status: NEW → ASSIGNED

Currently behavior:
Long push ./try fuzzy -q "'mochi":

estimates: Runs 974 tasks (945 selected, 29 dependencies)
estimates: Total task duration 12 days, 22:08:28
estimates: In the 10% percentile
estimates: Should take about 1:44:44 (Finished around 2021-12-16 19:10)

Shorter push ./try fuzzy -q "'mochi 'webgl":

estimates: Runs 202 tasks (181 selected, 21 dependencies)
estimates: Total task duration 1 day, 14:54:30
estimates: In the 36% percentile
estimates: Should take about 1:25:12 (Finished around 2021-12-16 18:55)

These should say either e.g. "90th percentile" or "top 10%".

Pushed by jgilbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a7fd9cfcf1e9
Improve clarity of try percentile estimates. r=ahal
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 97 Branch
Regressions: 1748357
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: