1554848 - 2.2% ts_paint_webext (windows7-32-shippable) regression on push ad38bcbe7879cf715df2a49840b62d1235689882

Status: VERIFIED FIXED

(Firefox :: General, defect, P1)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=5a721a7648f2db40785729ed8fc7c7444c1afcaf&tochange=ad38bcbe7879cf715df2a49840b62d1235689882

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

2% ts_paint_webext windows7-32-shippable opt e10s stylo 326.42 -> 333.59

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=21112

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Performance_sheriffing/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[Brian Grinstead [:bgrins]]

Were other platforms affected as well, or is this only win7-32?

Comment 2

[Brian Grinstead [:bgrins]]

What's the difference between ts_paint_webext and ts_paint? I don't see it listed in https://wiki.mozilla.org/Buildbot/Talos/Tests#ts_paint.

Comment 3

[Ionuț Goldan [:igoldan]]

(In reply to Brian Grinstead [:bgrins] from comment #1)

Were other platforms affected as well, or is this only win7-32?

Only Windows platforms got affected. This includes Windows 10.

Comment 4

[Ionuț Goldan [:igoldan]]

(In reply to Brian Grinstead [:bgrins] from comment #2)

What's the difference between ts_paint_webext and ts_paint? I don't see it listed in https://wiki.mozilla.org/Buildbot/Talos/Tests#ts_paint.

:kmag can answer that.

Comment 5

[Kris Maglione [:kmag]]

(In reply to Brian Grinstead [:bgrins] from comment #2)

What's the difference between ts_paint_webext and ts_paint? I don't see it listed in https://wiki.mozilla.org/Buildbot/Talos/Tests#ts_paint.

ts_paint_webext has an extra extension installed that's meant to test overhead from the extension framework.

Comment 6

[Brian Grinstead [:bgrins]]

It's not clear to me what in particular about browser.xhtml + the extension framework would trigger that's slower. The document load should be the same (using prototype cache), and ts_paint itself is fine. Maybe the extension framework is special casing XULDocument somehow, though I'm not seeing anything obvious at https://searchfox.org/mozilla-central/search?q=xul&path=toolkit%2Fcomponents%2Fextensions.

Comment 7

[Brian Grinstead [:bgrins]]

Brendan, do you have any ideas on the next step for debugging this? I can do some try pushes to gather profiles with and without the change but not sure where to go besides that.

Comment 8

[Kris Maglione [:kmag]]

If I had to guess, I'd say the most likely candidates for being affected by browser.xhtml would be the browser action and page action code. I'm not sure exactly why it would make a difference, though.

The sheriffs should be able to get you before and after profiles.

Comment 9

[Brian Grinstead [:bgrins]]

Pushes that should get profiles (gathered via ./mach try fuzzy -q 'win talos-g5' --gecko-profile --no-artifact):

• Current m-c: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a548ea9bba131cb8cb2f978a5bdf099045630814
• With backout: https://treeherder.mozilla.org/#/jobs?repo=try&revision=047204c4f74a369fba5b0e89d0d8079170a842b9

Comment 10

[Brian Grinstead [:bgrins]]

(In reply to Brian Grinstead [:bgrins] from comment #9)

Pushes that should get profiles (gathered via ./mach try fuzzy -q 'win talos-g5' --gecko-profile --no-artifact):

• Current m-c: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a548ea9bba131cb8cb2f978a5bdf099045630814
• With backout: https://treeherder.mozilla.org/#/jobs?repo=try&revision=047204c4f74a369fba5b0e89d0d8079170a842b9

Hm, looks like these didn't work. Ionuț, could you help get profiles before/after the regression (and let me know how you do it so I can do it next time)?

Comment 11

[Brian Grinstead [:bgrins]]

A couple of pushes from today:

• There's a low confidence improvement by using a sync domcontentloaded event: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=5cf85a61738a894836ecf791d2721a1828b6112e&newProject=try&newRevision=6de35e9b14c741cfc693804e942fea70075ef34f&framework=1&showOnlyImportant=1
• No change when using dummy.xhtml instead of dummy.xul: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=5cf85a61738a894836ecf791d2721a1828b6112e&newProject=try&newRevision=c8a70836428841df0ee6fde55620e0797e5e8827&framework=1&showOnlyImportant=1

Comment 12

[Ionuț Goldan [:igoldan]]

(In reply to Brian Grinstead [:bgrins] from comment #7)

Brendan, do you have any ideas on the next step for debugging this? I can do some try pushes to gather profiles with and without the change but not sure where to go besides that.

I tried to run the profiles, but I get permafails on Windows.

Comment 13

[Brendan Dahl [:bdahl]]

Attachment: Bug 1554848 - Use sync DOMContentLoaded when using prototype cache. r=bzbarsky

Some of the talos test timers seem to be affected when DOMContentLoaded
is not dispatched synchronously. This will also help when we load all XUL
as XHTML (bug 1550801), since certain tests rely on sync dispatch.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1a68a4a5a5b7
Use sync DOMContentLoaded when using prototype cache. r=bzbarsky

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/1a68a4a5a5b7