1929227 - 7.88% cnn ContentfulSpeedIndex (OSX) regression on Wed October 30 2024

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect, P2)

Description

[Treeherder Bug Filer]

Perfherder has detected a browsertime performance regression from push ce9f60c590954891fe6bc075111648e143cdecc6. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
64%	google-slides loadtime	macosx1015-64-shippable-qr	bytecode-cached fission warm webrender	734.72 -> 1,203.02	Before/After
58%	google-slides SpeedIndex	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	398.16 -> 628.71	Before/After
57%	google-slides loadtime	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	910.33 -> 1,430.26	Before/After
49%	google-slides largestContentfulPaint	macosx1015-64-shippable-qr	bytecode-cached fission warm webrender	955.47 -> 1,424.56	Before/After
48%	google-slides PerceptualSpeedIndex	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	467.67 -> 694.32	Before/After
46%	google-slides ContentfulSpeedIndex	macosx1015-64-shippable-qr	bytecode-cached fission warm webrender	659.86 -> 962.50	Before/After
46%	google-slides ContentfulSpeedIndex	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	761.57 -> 1,110.29	Before/After
45%	google-slides loadtime	windows11-64-shippable-qr	bytecode-cached fission warm webrender	584.13 -> 848.97	Before/After
43%	google-slides largestContentfulPaint	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	1,160.76 -> 1,658.57	Before/After
40%	google-slides SpeedIndex	windows11-64-shippable-qr	bytecode-cached fission warm webrender	311.52 -> 437.38	Before/After
...	...	...	...	...	...
5%	cnn FirstVisualChange	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	569.45 -> 600.04	Before/After
5%	cnn PerceptualSpeedIndex	windows11-64-shippable-qr	bytecode-cached fission warm webrender	301.40 -> 317.50	Before/After
5%	cnn ContentfulSpeedIndex	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	583.86 -> 614.60	Before/After
5%	fandom LastVisualChange	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	319.44 -> 333.97	Before/After
4%	cnn fcp	windows11-64-shippable-qr	bytecode-cached fission warm webrender	302.06 -> 315.41	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
33%	google-slides largestContentfulPaint	windows11-64-shippable-qr	bytecode-cached fission warm webrender	91.55 -> 61.00	Before/After
29%	google-slides fcp	windows11-64-shippable-qr	bytecode-cached fission warm webrender	82.62 -> 58.53	Before/After
28%	google-slides FirstVisualChange	macosx1015-64-shippable-qr	bytecode-cached fission warm webrender	169.29 -> 122.60	Before/After
24%	google-slides FirstVisualChange	windows11-64-shippable-qr	bytecode-cached fission warm webrender	108.83 -> 82.77	Before/After
23%	google-slides fcp	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	126.23 -> 97.47	Before/After
17%	google-slides FirstVisualChange	linux1804-64-shippable-qr	bytecode-cached fission warm webrender	142.58 -> 118.68	Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run all of these tests on try with ./mach try perf --alert 42412

The following documentation link provides more information about this command.

For more information on performance sheriffing please see our FAQ.

If you have any questions, please do not hesitate to reach out to [e-mail].

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1724236

Comment 2

[Bryan Thrall [:bthrall]]

Attachment: Bug 1929227 - Restore bytecode handling r=mgaudet!

It was accidentally removed in 7b334fe58713.

Comment 3

[Mayank Bansal]

Curious, why did the regressing bug bring large improvements to some of the google-slides metrics that measure page load like fcp,lcp, loadtime.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5b67007fef7
Restore bytecode handling r=mgaudet

Comment 5

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/a5b67007fef7

Comment 6

[Mayank Bansal]

Looks like the numbers are back to pre-regression: https://treeherder.mozilla.org/perfherder/alerts?id=42478 (Not validated by perf-sheriffs yet)

Comment 7

[Bryan Thrall [:bthrall]]

Bug 1931320 confirms the numbers are back to pre-regression.