1694578 - 0.09 - 0.18% installer size (osx-shippable) regression on push af7e4b09a4fb8d72b7945ece38b87a31e9b1c85c (Fri February 19 2021)

Status: RESOLVED FIXED

(Firefox :: Messaging System, defect, P1)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Perfherder has detected a build_metrics performance regression from push af7e4b09a4fb8d72b7945ece38b87a31e9b1c85c. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
0.18%	installer size		osx-shippable	nightly	85,792,188.67 -> 85,944,741.33
0.16%	installer size		osx-shippable	instrumented	114,154,368.17 -> 114,337,820.42
0.09%	installer size		osx-shippable	instrumented	114,170,267.42 -> 114,269,629.83

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1686343

Comment 2

[Ed Lee :Mardak]

This is expected for adding an animated image used during onboarding via a remote experiment. We can also be smarter by packaging it only for windows builds if we decide to keep it when turning it on by default in bug 1694257.

Comment 3

[Ed Lee :Mardak]

Attachment: Bug 1694578 - Package windows assets used remotely only for Windows r?pdahiya

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/21d5f47b6604
Package windows assets used remotely only for Windows r=pdahiya

Comment 5

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/21d5f47b6604

Comment 6

[Alexandru Ionescu (needinfo me) [:alexandrui]]

:Mardak, seems like the regression wasn't fixed yet https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&highlightedRevisions=21d5f47b6604&series=autoland,2259234,1,2&timerange=604800

Comment 7

[Ed Lee :Mardak]

The original regression for installer size osx-shippable instrumented was between Δ 99362.41 (+0.09%) and Δ 183452.25 (+0.16%).

This revision shows up as Δ -130295.00 (-0.1%) which is pretty close to the average of the absolute delta of the regression.

There seems to be a recent Δ 400983.79 (+0.35%) regression untriaged alert that could make the small percentage change improvement slightly smaller than the regressing percentage. https://treeherder.mozilla.org/perfherder/alerts?id=29310