1587759 - 2.97 - 146.11% raptor-tp6-* regression on push be9a6289486a6f366e431782b84a0c0633f8fec2 (Wed October 9 2019)

Status: RESOLVED FIXED

(Core :: DOM: Service Workers, defect, P2)

Description

[Marian Raiciof [:marauder]]

2.97 - 146.11% raptor-tp6-binast-instagram-firefox / raptor-tp6-binast-instagram-firefox loadtime / raptor-tp6-instagram-firefox / raptor-tp6-instagram-firefox loadtime / raptor-tp6-linkedin-firefox / raptor-tp6-linkedin-firefox fcp / raptor-tp6-linkedin-firefox loadtime / raptor-tp6-twitter-firefox / raptor-tp6-twitter-firefox fcp / raptor-tp6-twitter-firefox loadtime / raptor-tp6-youtube-firefox / raptor-tp6-youtube-firefox loadtime / raptor-tp6-youtube-firefox-cold / raptor-tp6-youtube-firefox-cold loadtime (linux64-shippable, linux64-shippable-qr, macosx1014-64-shippable, windows10-64-shippable, windows10-64-shippable-qr, windows7-32-shippable) regression on push be9a6289486a6f366e431782b84a0c0633f8fec2 (Wed October 9 2019)

---

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=be9a6289486a6f366e431782b84a0c0633f8fec2

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

146% raptor-tp6-twitter-firefox loadtime windows10-64-shippable-qr opt 328.71 -> 809.00
144% raptor-tp6-twitter-firefox loadtime windows10-64-shippable opt 326.96 -> 796.96
139% raptor-tp6-twitter-firefox loadtime windows7-32-shippable opt 321.88 -> 768.04
125% raptor-tp6-twitter-firefox loadtime macosx1014-64-shippable opt 618.92 -> 1,390.00
124% raptor-tp6-twitter-firefox loadtime linux64-shippable opt 363.62 -> 814.46
122% raptor-tp6-twitter-firefox loadtime linux64-shippable-qr opt 374.92 -> 833.92
41% raptor-tp6-instagram-firefox loadtime windows10-64-shippable opt 566.12 -> 799.92
37% raptor-tp6-instagram-firefox loadtime windows10-64-shippable-qr opt 571.00 -> 781.25
36% raptor-tp6-instagram-firefox loadtime windows7-32-shippable opt 578.92 -> 784.54
31% raptor-tp6-twitter-firefox windows10-64-shippable-qr opt 282.06 -> 368.47
30% raptor-tp6-twitter-firefox windows10-64-shippable opt 280.30 -> 363.87
29% raptor-tp6-linkedin-firefox fcp windows10-64-shippable-qr opt 457.62 -> 589.92
29% raptor-tp6-linkedin-firefox windows10-64-shippable-qr opt 574.46 -> 740.11
29% raptor-tp6-twitter-firefox windows7-32-shippable opt 275.48 -> 354.77
27% raptor-tp6-linkedin-firefox fcp linux64-shippable opt 427.83 -> 544.83
27% raptor-tp6-linkedin-firefox fcp windows7-32-shippable opt 444.25 -> 565.29
27% raptor-tp6-linkedin-firefox windows7-32-shippable opt 558.34 -> 710.05
27% raptor-tp6-twitter-firefox macosx1014-64-shippable opt 497.16 -> 632.24
26% raptor-tp6-linkedin-firefox windows10-64-shippable opt 577.54 -> 729.36
26% raptor-tp6-linkedin-firefox fcp windows10-64-shippable opt 460.17 -> 580.38
26% raptor-tp6-linkedin-firefox loadtime windows10-64-shippable-qr opt 1,239.54 -> 1,560.54
26% raptor-tp6-linkedin-firefox fcp linux64-shippable-qr opt 454.38 -> 570.92
25% raptor-tp6-twitter-firefox linux64-shippable opt 318.66 -> 399.78
25% raptor-tp6-twitter-firefox linux64-shippable-qr opt 330.14 -> 413.61
25% raptor-tp6-linkedin-firefox macosx1014-64-shippable opt 927.91 -> 1,160.89
25% raptor-tp6-linkedin-firefox linux64-shippable opt 545.64 -> 681.00
24% raptor-tp6-linkedin-firefox loadtime windows7-32-shippable opt 1,217.38 -> 1,514.88
24% raptor-tp6-linkedin-firefox loadtime windows10-64-shippable opt 1,253.88 -> 1,550.38
23% raptor-tp6-linkedin-firefox linux64-shippable-qr opt 580.76 -> 711.82
21% raptor-tp6-youtube-firefox loadtime windows10-64-shippable-qr opt 791.77 -> 960.00
21% raptor-tp6-youtube-firefox loadtime windows7-32-shippable opt 791.94 -> 956.04
20% raptor-tp6-youtube-firefox loadtime windows10-64-shippable opt 789.62 -> 947.96
18% raptor-tp6-youtube-firefox loadtime windows10-64-shippable opt 810.54 -> 959.50
18% raptor-tp6-youtube-firefox loadtime linux64-shippable opt 843.75 -> 998.25
16% raptor-tp6-linkedin-firefox loadtime macosx1014-64-shippable opt 2,217.96 -> 2,570.58
16% raptor-tp6-instagram-firefox windows10-64-shippable-qr opt 242.75 -> 280.67
16% raptor-tp6-instagram-firefox windows10-64-shippable opt 242.77 -> 280.62
15% raptor-tp6-youtube-firefox loadtime linux64-shippable-qr opt 887.92 -> 1,023.75
15% raptor-tp6-linkedin-firefox loadtime linux64-shippable opt 1,254.17 -> 1,437.79
14% raptor-tp6-instagram-firefox windows7-32-shippable opt 242.19 -> 275.02
11% raptor-tp6-linkedin-firefox loadtime linux64-shippable-qr opt 1,332.92 -> 1,485.21
10% raptor-tp6-binast-instagram-firefox windows10-64-shippable-qr opt 223.38 -> 245.35
9% raptor-tp6-binast-instagram-firefox linux64-shippable-qr opt 230.87 -> 252.08
9% raptor-tp6-binast-instagram-firefox windows10-64-shippable opt 223.61 -> 243.57
9% raptor-tp6-instagram-firefox linux64-shippable opt 247.52 -> 268.73
9% raptor-tp6-binast-instagram-firefox windows7-32-shippable opt 222.08 -> 241.07
8% raptor-tp6-instagram-firefox linux64-shippable-qr opt 250.89 -> 272.11
7% raptor-tp6-youtube-firefox-cold loadtime macosx1014-64-shippable opt 1,536.71 -> 1,639.17
7% raptor-tp6-binast-instagram-firefox linux64-shippable opt 231.43 -> 246.67
6% raptor-tp6-youtube-firefox linux64-shippable-qr opt 558.95 -> 594.72
6% raptor-tp6-youtube-firefox windows7-32-shippable opt 526.57 -> 559.93
6% raptor-tp6-youtube-firefox windows10-64-shippable opt 528.58 -> 561.94
6% raptor-tp6-twitter-firefox fcp windows10-64-shippable-qr opt 283.50 -> 299.33
5% raptor-tp6-youtube-firefox-cold windows10-64-shippable opt 613.88 -> 646.39
5% raptor-tp6-youtube-firefox windows10-64-shippable-qr opt 526.74 -> 552.56
4% raptor-tp6-twitter-firefox fcp windows10-64-shippable opt 282.71 -> 295.33
4% raptor-tp6-binast-instagram-firefox loadtime windows10-64-shippable-qr opt 432.69 -> 450.38
4% raptor-tp6-youtube-firefox linux64-shippable opt 558.05 -> 580.46
4% raptor-tp6-binast-instagram-firefox loadtime windows10-64-shippable opt 432.96 -> 448.88
3% raptor-tp6-twitter-firefox fcp linux64-shippable-qr opt 331.42 -> 342.58
3% raptor-tp6-twitter-firefox fcp linux64-shippable opt 321.19 -> 331.67
3% raptor-tp6-binast-instagram-firefox loadtime linux64-shippable-qr opt 458.12 -> 471.83
3% raptor-tp6-binast-instagram-firefox loadtime windows7-32-shippable opt 422.83 -> 435.38

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=23395

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/TestEngineering/Performance/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

It looks like we do have some performance coverage of ServiceWorkers after all!

Some performance regressions were expected in ServiceWorker-using sites but twitter is a bit worse than hoped for. We can probably attempt to address this by having the interception logic trigger a preload of the underlying DOM Cache storage to reduce ServiceWorker spin-up latency. We should also see what altering the process selection logic does in terms of explicitly picking the same process or a separate process.

To state the context more explicitly:

• This re-architecture was always expected to regress ServiceWorker-using performance in the name of correctness and simplicity.
• Fission and SharedArrayBuffer re-enabling are predicated on the parent-intercept mode of operation.
• Most of the performance improvements we will be making initially are things that are potentially uplift-friendly because they're all largely about preloading.

Comment 2

[Perry Jiang [:perry] [no longer employee, use ni?]]

Clearing ni? due to context given by Andrew.

Comment 3

[Pascal Chevrel:pascalc]

Marking as disabled for 71 as this is caused by a nightly-only pref switch.

Comment 5

[Jim Mathies [:jimm]]

caused by a nightly only feature

Comment 6

[Vicky Chin [:vchin]]

Since this is riding the train I'd like to understand where we currently stand on the significant performance regressions to top sites. Have we landed any fixes to mitigate? Are there bugs filed with follow on improvements planned? If the numbers are still where they were from comment 3, I would say this is fx73 blocking bug from a performance perspective.

Comment 7

[Jens Stutte [:jstutte]]

:ytausky is analyzing the specific twitter regression as well as general performance improvements. He will report back here and link the relative bugs.

Said this, we have some strong functional dependencies on bug 1588154 for FF 73 that might outweigh the performance regression for the time being, hoping obviously that this time is very short.

Comment 8

[Yaron Tausky [:ytausky]]

So far I found out the following things:

1. When running in child intercept mode the page loads a set A of resources, fires the load event, and then loads a set B of two additional scripts.
2. When running in parent intercept mode the page loads A, then B, then runs the scripts in B, and they cause the page to load a set C of more resources. Once C is loaded, the load event is fired.
3. The page causes the scripts in B to be loaded by setting a timeout of 0 with a handler that inserts script tags into the document.

So the change here is actually functional, since work that used to be done after the load event is now done before it. The question is which of the two behaviors is the correct one. (Chrome taking a similar amount of time to run this benchmark suggests that the new behavior might be the correct one.) Anne, maybe you could help us determine which one it is?

Comment 9

[Anne (:annevk)]

As far as I can tell all script elements inserted into the document before the "main load event" is dispatched end up delaying it. This follows from

Once it is set, either to a script in the case of success or to null in the case of failure, the fetching algorithms will note that the script is ready, which can trigger other actions. The user agent must delay the load event of the element's node document until the script is ready.

and algorithms that refer those concepts. There are some holes in this definition though, e.g., https://github.com/whatwg/html/issues/5217, but the intent is that script elements that will end up executing delay the "main load event". Hope that helps.

Comment 10

[Yaron Tausky [:ytausky]]

I filed bug 1609936 regarding the unexpected redirections I also see in the profile.

Comment 11

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Anne (:annevk) from comment #9)

As far as I can tell all script elements inserted into the document before the "main load event" is dispatched end up delaying it.

Right; anything added to the page (images, scripts) before the load event fires causes them to be added to the LoadGroup, and the load event doesn't fire until everything in the LoadGroup has resolved. (deeply paraphrased from various specs). This was the reason for landing setTimeout deferral-during-load (which defers setTimeouts until we fire Idle when we're loading). For many pages this pushes a bunch of ancillary loads/scripts to be post-load-event, greatly improving loadtime (though generally having little impact on visual metrics, other than the throbber stops (and stop/reload switches) far earlier. See bug 1270059

This is interacting with that code, and presumably causing Idle to occur before load, letting the setTimeouts get in and add resources. This will be inherently timing and machine-sensitive, though it's unclear how much (a (fast? slwo spinny disk? long RTT?) machine that reliably got Idle anyways before load will be unaffected)

Comment 12

[Ryan VanderMeulen [:RyanVM]]

This has been pretty heavily analyzed by the engineering and perf teams and the conclusion is that this isn't an actual user-perceptible regression. Bug 1407276 should mitigate it regardless, but that's unlikely to be uplifted to 74.

Comment 13

[Jens Stutte [:jstutte]]

So we won't uplift bug 1407276 to 75 beta? Then this can be closed for 76.