1086539 - 3-25% a11y/dromaeo dom Windows regression on inbound (v.36) Oct 16th from revision a758ca998666

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

We have a series of regressions, most concerning are the a11y ones:
http://54.215.155.53:8080/alerts.html?rev=a758ca998666&table=1

I did some retriggers:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&fromchange=4bdbd3e97f00&tochange=35da3c99e658&jobname=mozilla-inbound%20pgo%20talos

the culprit is clearly:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=a758ca998666

Here are a list of regressions:
Tp5 Optimized Responsiveness
winxp -6.15%

Tp5 Optimized
win7 -5%
win8 -4.71%

Dromaeo (DOM)
win8 -3.59%

Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes)
win7 -25.2%

a11y Row Major MozAfterPaint
win7 -11.8%
win8 -11.9%

I am not sure how this related to bug 1084462, maybe it doesn't.

Comment 1

[Mike Hommey [:glandium]]

(In reply to Joel Maher (:jmaher) from comment #0)
> We have a series of regressions, most concerning are the a11y ones:
> http://54.215.155.53:8080/alerts.html?rev=a758ca998666&table=1
> 
> I did some retriggers:
> https://tbpl.mozilla.org/?tree=Mozilla-
> Inbound&fromchange=4bdbd3e97f00&tochange=35da3c99e658&jobname=mozilla-
> inbound%20pgo%20talos
> 
> the culprit is clearly:
> http://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=a758ca998666
> 
> Here are a list of regressions:
> Tp5 Optimized Responsiveness
> winxp -6.15%
> 
> Tp5 Optimized
> win7 -5%
> win8 -4.71%
> 
> Dromaeo (DOM)
> win8 -3.59%
> 
> Tp5 No Network Row Major MozAfterPaint (Main Startup File IO Bytes)
> win7 -25.2%

This one was fixed by followups (and is in fact now 1MB lower than before)

> a11y Row Major MozAfterPaint
> win7 -11.8%
> win8 -11.9%
> 
> I am not sure how this related to bug 1084462, maybe it doesn't.

The same as that bug applies: we don't really care if the PGO builds are not impacted.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

this set of regressions are on PGO only, and yes- I have verified the startup file io stuff is cleaned up!

Comment 3

[Mike Hommey [:glandium]]

Could you compare those results with those before bug 1061335? For instance, it seems the a11y results are now about the level they were before we switched to VS2013.

Comment 4

[Mike Hommey [:glandium]]

Also, dromaeo still seems to be above the values we have before VS2013.

Comment 5

[Mike Hommey [:glandium]]

Maybe we should push a try build with vs2010 and folding undone and compare that?

Comment 6

[Mike Hommey [:glandium]]

The Tp5 Optimized Responsiveness regression seems like a red herring, seeing the overall graph.

Comment 7

[Mike Hommey [:glandium]]

Finally, Tp5 optimized looks better now than with vs2010.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

yeah, the delta from a few days earlier before the packing of the libraries yields a net win for tp5 optimized and tp5o responsiveness.

the main startup io file bytes is about equal to a few days earlier.

This leaves overall regressions as:
dromaeo dom (this is a reverse test, so higher is not better, we want lower values)
a11y

Comment 9

[xunxun]

(In reply to Mike Hommey [:glandium] from comment #7)
> Finally, Tp5 optimized looks better now than with vs2010.

Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it again?

According to my some earlier tests, merging js/gkmedias to xul may have some regression in some benchmark.

Comment 10

[xunxun]

(In reply to xunxun from comment #9)
> (In reply to Mike Hommey [:glandium] from comment #7)
> > Finally, Tp5 optimized looks better now than with vs2010.
> 
> Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> again?
> 
> According to my some earlier tests, merging js/gkmedias to xul may have some
> regression in some benchmark.

Sorry, I mean bug 609976

Comment 11

[Mike Hommey [:glandium]]

(In reply to xunxun from comment #9)
> (In reply to Mike Hommey [:glandium] from comment #7)
> > Finally, Tp5 optimized looks better now than with vs2010.
> 
> Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> again?
> 
> According to my some earlier tests, merging js/gkmedias to xul may have some
> regression in some benchmark.

Do you realize this is what this bug is about?

Comment 12

[xunxun]

(In reply to Mike Hommey [:glandium] from comment #11)
> (In reply to xunxun from comment #9)
> > (In reply to Mike Hommey [:glandium] from comment #7)
> > > Finally, Tp5 optimized looks better now than with vs2010.
> > 
> > Can you try to revert bug 922912 and bug 6066976, using VC2013 to build it
> > again?
> > 
> > According to my some earlier tests, merging js/gkmedias to xul may have some
> > regression in some benchmark.
> 
> Do you realize this is what this bug is about?

Is there any inappropriate words I said?
I mean a758ca998666 is the js merging commit, which may cause some benchmark regression when I tested PGO build, so may also cause a11y/dromaeo dom Windows regression.

Comment 13

[(Away)]

Bug 1084162 brought the a11y test down to slightly better than before the VS2013 changes began. It also helped a little bit on Dromaeo, but that was already better than pre-VS2013.

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

all is better except for win7 a11y:
http://graphs.mozilla.org/graph.html#tests=%5B%5B223,63,25%5D%5D&sel=1412683809299,1415275809299&displayrange=30&datatype=running

in this case we were ~155 before and now ~160

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

Is there any additional action to take here?  I would  like to determine what we are doing with the 1 remaining test case.

Comment 16

[Avi Halachmi (:avih)]

Considering we're only left with a single test which regressed about 3%, I don't think we should be doing anything else here. Let's close it.