1454584 - 2.7% rasterflood_svg (windows7-32) regression on push ab7ad29703d5 (Tue Apr 10 2018)

Status: RESOLVED INVALID

(Core :: DOM: Core & HTML, defect, P3)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=7bd226b1ef552ce282f1468cfff42d72e8d2c7c7&tochange=ac866c77cec09dfbf94fcef12c3e88461fe65212

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  3%  rasterflood_svg windows7-32 opt e10s stylo     10,160.83 -> 10,435.50


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=12666

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

If I do a backout from this changeset [1] and then compare the two builds, I can cancel the above regression [2].
This shows that bug 1451790 is the source of it.

Do we accept this or should we tweak it a little?

[1] https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=7bd226b1ef552ce282f1468cfff42d72e8d2c7c7&tochange=ac866c77cec09dfbf94fcef12c3e88461fe65212
[2] https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=cf1bb54c4cf0&newProject=try&newRevision=28dcfbb96330&framework=1&filter=rasterflood_svg

Comment 2

[Tom Ritter [:tjr]]

Hm. In Bug 1445243 we disabled timer precision reduction on Talos. So changing the pref value or backing out the change shouldn't have any effect. And there were no other changes in the patch....

Comment 3

[Tom Ritter [:tjr]]

How confident are you that reverting clears the regression? If so, I think we may need to go investigate why Talos isn't applying prefs correctly...

Comment 4

[Ionuț Goldan [:igoldan]]

(In reply to Tom Ritter [:tjr] from comment #3)
> How confident are you that reverting clears the regression? If so, I think
> we may need to go investigate why Talos isn't applying prefs correctly...

I'm confident. I backed out each bug separately from the patch [1], and only bug 1451790 canceled the regression.
Other didn't influence it in any way.

[1] https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=7bd226b1ef552ce282f1468cfff42d72e8d2c7c7&tochange=ac866c77cec09dfbf94fcef12c3e88461fe65212

Comment 5

[Tom Ritter [:tjr]]

Hm.  I wasn't sure where to start, so I tried to reproduce it and wasn't able to:

https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=56ca7449d364202ac2f106a4324554f7b35675e5&newProject=try&newRevision=c398f79e273bf7016a8610742772cfaf563c0d9a&framework=1

Also, I noticed this was opt only; it didn't occur on pgo? That may make it less of a concern also?

Comment 6

[Ionuț Goldan [:igoldan]]

(In reply to Tom Ritter [:tjr] from comment #5)
> Hm.  I wasn't sure where to start, so I tried to reproduce it and wasn't
> able to:
> 
> https://treeherder.mozilla.org/perf.html#/
> compare?originalProject=try&originalRevision=56ca7449d364202ac2f106a4324554f7
> b35675e5&newProject=try&newRevision=c398f79e273bf7016a8610742772cfaf563c0d9a&
> framework=1
> 
> Also, I noticed this was opt only; it didn't occur on pgo? That may make it
> less of a concern also?

This happened on mozilla-beta. The OPT buids here are actually PGO builds.

Comment 7

[Tom Ritter [:tjr]]

I cuoldn't get a difference for PGO on beta either: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=a36a2a865d926a3f0c7b96233ccbb63ade911e68&newProject=try&newRevision=f8f4f76e97b1d27f383b97f4cb881f25002dee65&framework=1

Comment 8

[Ionuț Goldan [:igoldan]]

(In reply to Tom Ritter [:tjr] from comment #7)
> I cuoldn't get a difference for PGO on beta either:
> https://treeherder.mozilla.org/perf.html#/
> compare?originalProject=try&originalRevision=a36a2a865d926a3f0c7b96233ccbb63a
> de911e68&newProject=try&newRevision=f8f4f76e97b1d27f383b97f4cb881f25002dee65&
> framework=1

I tried to re-reproduce this my self, without luck. I'm attempting a second bisect.

Comment 9

[Ionuț Goldan [:igoldan]]

Second bisect: https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=ac5af2f6a9e1&newProject=try&newRevision=0cec57594bcf&framework=1&filter=rasterflood_

I've backed out bug 1451790 from the original push: https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=7bd226b1ef552ce282f1468cfff42d72e8d2c7c7&tochange=ac866c77cec09dfbf94fcef12c3e88461fe65212

I managed to validate again that there's indeed a regression, on both OPT & PGO builds.

Comment 10

[Tom Ritter [:tjr]]

D'oh.

I looked at commit dates but didn't dig into the tree...

https://hg.mozilla.org/mozilla-central/rev/22ca6f534784 puts the pref into talos/config.py

It's not in -beta as of this bisect: https://hg.mozilla.org/releases/mozilla-beta/file/ab7ad29703d5/testing/talos/talos/config.py




So ultimately, this regression is an artifact of me messing with how Talos records numbers, not changing the behavior of the browser. We should probably uplift https://hg.mozilla.org/mozilla-central/rev/22ca6f534784 to all branches you want Talos nmbers on.

Comment 11

[Tom Ritter [:tjr]]

Ah it went into -beta in April 26th: https://hg.mozilla.org/releases/mozilla-beta/rev/22ca6f534784

That's why we couldn't reproduce it with bisect and why I didn't see the problem originally (I thought I had checked -beta to see if the pref was there.)

Comment 12

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Closing as INVALID due to this being a test change that was not uplifted to beta. It's also over 2 years old so there's not a lot we could do at this time.