(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #17) > Browsertime: > https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&newProject=try&newRevision=c4a952e8e5c1d2ac146e0876e0fc015f60a59f59&originalSignature=2602001&newSignature=2602001&framework=13&originalRevision=a9236835fac366066055eacd3d1bb30a9667290c > > Keep in mind the alert is on **windows10-64-shippable** the test name is **google-search opt *cold*** and the stubtest is **loadtime** After increasing the number of retriggers from 7 to 16, the improvement in this metric went from 17% to 3.6%. The results of individual test runs are clustered around three values (~380, ~720, and ~1050) both before and after, and the difference in the average seems to be due to variations in which value individual tests hit, which get smaller as the number of test runs increases. So, I don't think the patch is having a systematic effect here.
Bug 1650398 Comment 22 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #17) > Browsertime: > https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=try&newProject=try&newRevision=c4a952e8e5c1d2ac146e0876e0fc015f60a59f59&originalSignature=2602001&newSignature=2602001&framework=13&originalRevision=a9236835fac366066055eacd3d1bb30a9667290c > > Keep in mind the alert is on **windows10-64-shippable** the test name is **google-search opt *cold*** and the stubtest is **loadtime** After increasing the number of retriggers from 7 to 16, the improvement in this metric went from 17% to 3.6%. The results of individual test runs are clustered around three values (~380, ~720, and ~1050) both before and after, and the difference in the average seems to be due to variations in which value individual tests hit; this difference gets smaller as the number of test runs increases. So, I don't think the patch is having a systematic effect here.