1863568 - 0.18 - 0.14% installer size / installer size (OSX) regression on Thu November 2 2023

Status: ASSIGNED

(Core :: DOM: Web Authentication, defect, P3)

Description

[Alex Finder]

Perfherder has detected a build_metrics performance regression from push 7e810f9a69e38c059df450641e997ca3b054640e. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)
0.18%	installer size	osx-aarch64-shippable	aarch64 nightly	83,956,842.92 -> 84,109,493.83
0.14%	installer size	osx-shippable	nightly	88,117,832.12 -> 88,241,274.83

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1854908

Comment 2

[John Schanck [:jschanck]]

I could use some help determining if the size of this regression is consistent with the FTL changes that were made. There's not a lot of new code, and there aren't any new dependencies, so I'm struggling to see what's causing the installer size increase. If the regression is due to FTL changes, then we can remove some strings that are primarily for debugging.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

:msirringhaus, since you are the author of the regressor, bug 1854908, could you take a look?

For more information, please visit BugBot documentation.

Comment 4

[msirringhaus]

I can't get to it this week, but I'll look into it first thing next week.

Comment 5

[msirringhaus]

So, the size increase seems to be around 120kB?
The locales only make up 7k, the about-page itself (html, css, js) 32k (all uncompressed!).
Even counting the tests section which has 40k wouldn't get us up to those ~120k. And the tests-section shouldn't even be packaged. And although I'm not 100% certain, I doubt all of those files go into the installer uncompressed.
We also now have 2 new autogenerated bindings and are using one more function from auth-rs, which previously probably got stripped away. But none of those should have an effect anywhere near this.

I'll try to more deeply understand the packaging process, but simply judging from the raw input data, I have honestly no idea how this submission could cause an increase this large.

Comment 6

[msirringhaus]

Also, looking at the graphs a bit more: The installer size went up for a few days, then decreased again to the size before this submission, even though this submission was not backed out.
I'm very much leaning towards this being a false positive.
What would be the way to go here? A more deeper analysis of this submission, to rule it out even more certainly, or a deeper analysis of the perftest and all involved submissions (and infra changes) to pinpoint the root cause?
I'm only equipped to do the former, but this kind of research can't lead to a definitive answer, only increase the certainty of my guesswork.

Comment 7

[Alex Finder]

(In reply to msirringhaus from comment #6)

Also, looking at the graphs a bit more: The installer size went up for a few days, then decreased again to the size before this submission, even though this submission was not backed out.
I'm very much leaning towards this being a false positive.
What would be the way to go here? A more deeper analysis of this submission, to rule it out even more certainly, or a deeper analysis of the perftest and all involved submissions (and infra changes) to pinpoint the root cause?
I'm only equipped to do the former, but this kind of research can't lead to a definitive answer, only increase the certainty of my guesswork.

Hi Martin! Looking at the graph, there is a clear uptrend introduced after revision 7067e9e6ee6cd0bf12b36862ff2a062b6b8a9b89, and a visible comeback after revision 4712c7e340c1bdb63cc78d5b1b31b5652debb201, which could be attributed to a series of incremental improvements in the following push range. Please let me know via a needinfo if there other questions or concerns related to this ticket.