1734141 - Try estimate for jobs is unclear - "estimates: In the 79% percentile" -- is my push in the lowest or highest 79%?

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, defect, P3)

Description

[:Gijs (he/him)]

We could do with saying something like "24% of trypushes are faster/slower than yours".

Comment 1

[Kelsey Gilbert [:jgilbert]]

Attachment: Bug 1734141 - Improve clarity of try percentile estimates.

Avoid "percentile" for user output, since it's not clear if e.g. a 10th
percentile push duration is good or bad.
(Also the previous output incorrectly displayed "10% percentile" for a
90th percentile push duration estimate)

Comment 2

[Kelsey Gilbert [:jgilbert]]

Currently behavior:
Long push ./try fuzzy -q "'mochi":

estimates: Runs 974 tasks (945 selected, 29 dependencies)
estimates: Total task duration 12 days, 22:08:28
estimates: In the 10% percentile
estimates: Should take about 1:44:44 (Finished around 2021-12-16 19:10)

Shorter push ./try fuzzy -q "'mochi 'webgl":

estimates: Runs 202 tasks (181 selected, 21 dependencies)
estimates: Total task duration 1 day, 14:54:30
estimates: In the 36% percentile
estimates: Should take about 1:25:12 (Finished around 2021-12-16 18:55)

These should say either e.g. "90th percentile" or "top 10%".

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a7fd9cfcf1e9
Improve clarity of try percentile estimates. r=ahal

Comment 4

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/a7fd9cfcf1e9