990140 - 3.29% regression in dromaeo dom on osx 10.6 from inbound revision 0fb101fc5976

Status: RESOLVED WONTFIX

(Core :: DOM: Core & HTML, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

this came in today on dev.tree-management:
https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/_m63uJs_FsU

here is the graph:
http://graphs.mozilla.org/graph.html#tests=[[73,63,21]]&sel=none&displayrange=7&datatype=running

^ note, you need to look at the far right of the graph, there is another more obvious regression on the 28th, that is not what this bug is referencing.

I did some retriggers to verify we were not seeing noise:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=Rev4%20MacOSX%20Snow%20Leopard%2010.6%20mozilla-inbound%20talos%20dromaeojs&fromchange=0b596070fe54&tochange=12fbb6719ce9

in datazilla, you can see the subtests, specifically attr.html is where the regression is:
https://datazilla.mozilla.org/?start=1395679254&stop=1396284054&product=Firefox&repository=Mozilla-Inbound&os=mac&os_version=OS%20X%2010.6.8&test=dromaeo_dom&graph_search=0fb101fc5976&error_bars=false&project=talos

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

peterv, does this make sense for your push?

Comment 2

[Peter Van der Beken [:peterv]]

Not really, no. Also, don't think I'm seeing a regression/improvement when these patches first landed/were backed out and the only difference between that and the second landing is a change in a mochitest.

Comment 3

[Boris Zbarsky [:bzbarsky]]

Isn't this basically the same regression as bug 989414?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Boris, this regression is a slight dip for 10.6 on the 31st, and the ggc landing was on the 28th.  On the graph the larger dip is the ggc one and this is a slight shift in numbers on the 31st.

Looking back over the retriggers, we clearly dip ~20ms on revision 0fb101fc5976, although there was another ~10ms dip shortly before this (file regression bug 990233).

My theory is this did have a slight regression initially (although ggc landed around then which caused the bigger regression), and this is now triggering alerts because of the cumulative effect of the ~10ms and this ~20ms.

Avi, this is another example of a 10.6 only regression.  I know we care more about large regressions on 10.6, but your opinion here would be valued.

Comment 5

[Avi Halachmi (:avih)]

I've expressed my perspective in bug 985575, and Andreas Gal concurred in bug 990233 comment 10.

TL:DR; We should not care much about small 10.6-only regressions unless they're likely to indicate a bigger issue which could also affect other platforms.

3% counts as a minor regression in my book, so if it's 10.6 only and doesn't raise a flag with anyone as indicating a bigger issue, IMO let it be.

Comment 6

[Boris Zbarsky [:bzbarsky]]

> we clearly dip ~20ms on revision 0fb101fc5976

Alright.  I suppose it's theoretically possible that the outerization changes are not being optimized as well as we would have thought by the compiler...  In theory.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

as per comment 5, and discussion in bug 990490, we should weigh looking into this with how useful our time can be spent.  As this is only 10.6, this makes it less of a priority.  If the change in question has no obvious issue or optimization to be made, I recommend resolving this as a wontfix.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

Are we going to investigate this or accept this regression?

Comment 9

[Boris Zbarsky [:bzbarsky]]

I don't think this is worth investigating.