Closed Bug 1674317 Opened 6 months ago Closed 5 months ago

94.91 - 140.43% about_newtab_with_snippets (linux64-shippable, macosx1014-64-shippable, windows10-64-shippable, windows7-32-shippable) regression on push 9dfa75f6ec73f10849e81c214271f8dd4a20ad45 (Sat October 24 2020)

Categories

(Firefox :: New Tab Page, defect, P3)

Firefox 84
defect

Tracking

()

RESOLVED FIXED
84 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox82 --- unaffected
firefox83 --- unaffected
firefox84 --- fixed

People

(Reporter: alexandrui, Assigned: mconley, NeedInfo)

References

(Blocks 1 open bug, Regression)

Details

(4 keywords)

Attachments

(1 file)

Perfherder has detected a talos performance regression from push 9dfa75f6ec73f10849e81c214271f8dd4a20ad45. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
140% about_newtab_with_snippets windows10-64-shippable e10s stylo 29.11 -> 70.00
140% about_newtab_with_snippets windows7-32-shippable e10s stylo 28.08 -> 67.33
116% about_newtab_with_snippets linux64-shippable e10s stylo 30.72 -> 66.29
95% about_newtab_with_snippets macosx1014-64-shippable e10s stylo 36.37 -> 70.90

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
26% startup_about_home_paint_realworld_webextensions startup_about_home_paint_realworld_webextensions windows7-32-shippable e10s stylo 718.17 -> 529.58
25% startup_about_home_paint_realworld_webextensions startup_about_home_paint_realworld_webextensions windows10-64-shippable e10s stylo 712.42 -> 533.25
11% startup_about_home_paint_realworld_webextensions startup_about_home_paint_realworld_webextensions linux64-shippable e10s stylo 685.08 -> 610.75
11% startup_about_home_paint_realworld_webextensions startup_about_home_paint_realworld_webextensions linux64-shippable e10s stylo 685.17 -> 611.33

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(mconley)

Because 2cd3d250088 broke the builds for 3 revisions, I had to make a local backout and push to try to be able to see the results.

  • this try push corresponds to da718fa047b which is the last good which mesaures about 33 for about_newtab_with_snippets opt e10s stylo
  • this try push corresponds to 9dfa75f6ec which is the revision corresponding to Bug 1672517 that seems to be introducing the regression. It measures about 72 for about_newtab_with_snippets opt e10s stylo

The comparison is here. I'm not sure if the link contains the filtering, but if you search for about_newtab_with_snippets opt e10s stylo you'll see a regression of over 110%.

Component: Performance → New Tab Page
Product: Testing → Firefox

Set release status flags based on info from the regressing bug 1672517

Looking.

Assignee: nobody → mconley

This is a real regression. The problem here appears to be that when we load the first about:home from the about:home startup cache, the scripts that are loaded for it aren't inserted into the ScriptPreloader cache. This means that subsequent loads of about:home / about:newtab don't hit the cache, which results in slower times to load the page.

I'm discussing solutions with the JS team right now.

Flags: needinfo?(mconley)

The JS team is working on a big sweeping change to how JS caching works in the browser, so what tcampbell recommended I do is workaround this issue by using the subscript loader again for the about:home startup cache, and file a bug to remove that hackery and use the shiny new goodness when it's ready (~6 months).

Using the subscript loader is currently our best way of getting the loaded scripts recorded for entry
in the ScriptPreloader which is used to cache startup-related JavaScript within the startup window.

Originally, we were using <script> tags to load those scripts, but unfortunately, that loading mechanism
does not qualify for being noted by the ScriptPreloader.

This is a workaround until the Stencil project by the SpiderMonkey team Phase 2 is complete (see bug 1663956),
which should allow us to more easily note scripts to cache at runtime.

This also means we weren't hitting the ScriptPreloader cache when we were loading the scripts from the about:home startup cache, nor for new tabs / new windows. I'm glad our internal benchmark caught this!

Priority: -- → P3
Pushed by mconley@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/cf8e07fcfaa7
Load about:home scripts via the subscript loader when hitting the about:home startup cache. r=k88hudson
Pushed by mconley@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3d345090ddec
Load about:home scripts via the subscript loader when hitting the about:home startup cache. r=k88hudson
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 84 Branch

== Change summary for alert #27535 (as of Fri, 06 Nov 2020 05:33:41 GMT) ==

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
56% about_newtab_with_snippets windows7-32-shippable e10s stylo 65.94 -> 29.17
54% about_newtab_with_snippets linux64-shippable-qr e10s stylo webrender-sw 66.56 -> 30.39
54% about_newtab_with_snippets linux64-shippable-qr e10s stylo webrender-sw 66.23 -> 30.30
54% about_newtab_with_snippets windows10-64-shippable-qr e10s stylo webrender-sw 67.53 -> 31.31
53% about_newtab_with_snippets windows10-64-shippable e10s stylo 66.73 -> 31.13
52% about_newtab_with_snippets linux64-shippable e10s stylo 65.56 -> 31.21
9% rasterflood_svg windows10-64-shippable-qr e10s stylo webrender-sw 10,794.98 -> 9,820.63
8% rasterflood_svg linux64-shippable-qr e10s stylo webrender-sw 10,540.22 -> 9,737.55
2% startup_about_home_paint_realworld_webextensions startup_about_home_paint_realworld_webextensions linux64-shippable-qr e10s stylo webrender 872.75 -> 853.83

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=27535

You need to log in before you can comment on or make changes to this bug.