3-25% a11y/dromaeo dom Windows regression on inbound (v.36) Oct 16th from revision a758ca998666

RESOLVED FIXED

Status

Firefox Build System
General
RESOLVED FIXED
4 years ago
3 months ago

People

(Reporter: jmaher, Unassigned)

Tracking

({perf, regression})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [talos_regression])

(Reporter)

Description

4 years ago
We have a series of regressions, most concerning are the a11y ones:
http://54.215.155.53:8080/alerts.html?rev=a758ca998666&table=1

I did some retriggers:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&fromchange=4bdbd3e97f00&tochange=35da3c99e658&jobname=mozilla-inbound%20pgo%20talos

the culprit is clearly:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=a758ca998666

Here are a list of regressions:
Tp5 Optimized Responsiveness
winxp -6.15%

Tp5 Optimized
win7 -5%
win8 -4.71%

Dromaeo (DOM)
win8 -3.59%

Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes)
win7 -25.2%

a11y Row Major MozAfterPaint
win7 -11.8%
win8 -11.9%

I am not sure how this related to bug 1084462, maybe it doesn't.
(In reply to Joel Maher (:jmaher) from comment #0)
> We have a series of regressions, most concerning are the a11y ones:
> http://54.215.155.53:8080/alerts.html?rev=a758ca998666&table=1
> 
> I did some retriggers:
> https://tbpl.mozilla.org/?tree=Mozilla-
> Inbound&fromchange=4bdbd3e97f00&tochange=35da3c99e658&jobname=mozilla-
> inbound%20pgo%20talos
> 
> the culprit is clearly:
> http://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=a758ca998666
> 
> Here are a list of regressions:
> Tp5 Optimized Responsiveness
> winxp -6.15%
> 
> Tp5 Optimized
> win7 -5%
> win8 -4.71%
> 
> Dromaeo (DOM)
> win8 -3.59%
> 
> Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes)
> win7 -25.2%

This one was fixed by followups (and is in fact now 1MB lower than before)

> a11y Row Major MozAfterPaint
> win7 -11.8%
> win8 -11.9%
> 
> I am not sure how this related to bug 1084462, maybe it doesn't.

The same as that bug applies: we don't really care if the PGO builds are not impacted.
(Reporter)

Comment 2

4 years ago
this set of regressions are on PGO only, and yes- I have verified the startup file io stuff is cleaned up!
Could you compare those results with those before bug 1061335? For instance, it seems the a11y results are now about the level they were before we switched to VS2013.
Also, dromaeo still seems to be above the values we have before VS2013.
Maybe we should push a try build with vs2010 and folding undone and compare that?
The Tp5 Optimized Responsiveness regression seems like a red herring, seeing the overall graph.
Finally, Tp5 optimized looks better now than with vs2010.
(Reporter)

Comment 8

4 years ago
yeah, the delta from a few days earlier before the packing of the libraries yields a net win for tp5 optimized and tp5o responsiveness.

the main startup io file bytes is about equal to a few days earlier.

This leaves overall regressions as:
dromaeo dom (this is a reverse test, so higher is not better, we want lower values)
a11y
Summary: 3-25% Tp5/a11y/dromaeo dom Windows regression on inbound (v.36) Oct 16th from revision a758ca998666 → 3-25% a11y/dromaeo dom Windows regression on inbound (v.36) Oct 16th from revision a758ca998666

Comment 9

4 years ago
(In reply to Mike Hommey [:glandium] from comment #7)
> Finally, Tp5 optimized looks better now than with vs2010.

Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it again?

According to my some earlier tests, merging js/gkmedias to xul may have some regression in some benchmark.

Comment 10

4 years ago
(In reply to xunxun from comment #9)
> (In reply to Mike Hommey [:glandium] from comment #7)
> > Finally, Tp5 optimized looks better now than with vs2010.
> 
> Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> again?
> 
> According to my some earlier tests, merging js/gkmedias to xul may have some
> regression in some benchmark.

Sorry, I mean bug 609976
(In reply to xunxun from comment #9)
> (In reply to Mike Hommey [:glandium] from comment #7)
> > Finally, Tp5 optimized looks better now than with vs2010.
> 
> Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> again?
> 
> According to my some earlier tests, merging js/gkmedias to xul may have some
> regression in some benchmark.

Do you realize this is what this bug is about?

Comment 12

4 years ago
(In reply to Mike Hommey [:glandium] from comment #11)
> (In reply to xunxun from comment #9)
> > (In reply to Mike Hommey [:glandium] from comment #7)
> > > Finally, Tp5 optimized looks better now than with vs2010.
> > 
> > Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> > again?
> > 
> > According to my some earlier tests, merging js/gkmedias to xul may have some
> > regression in some benchmark.
> 
> Do you realize this is what this bug is about?

Is there any inappropriate words I said?
I mean a758ca998666 is the js merging commit, which may cause some benchmark regression when I tested PGO build, so may also cause a11y/dromaeo dom Windows regression.
Bug 1084162 brought the a11y test down to slightly better than before the VS2013 changes began. It also helped a little bit on Dromaeo, but that was already better than pre-VS2013.
(Reporter)

Comment 15

3 years ago
Is there any additional action to take here?  I would  like to determine what we are doing with the 1 remaining test case.
Considering we're only left with a single test which regressed about 3%, I don't think we should be doing anything else here. Let's close it.
(Reporter)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED

Updated

3 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.