978878 - 3-4% regression on tsvgx for android on mozilla-central

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

and talos alerts keep finding more stuff:
https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/_0YELL43jfE

here is a definition of the tests:
https://wiki.mozilla.org/Buildbot/Talos/Tests#tsvg.2C_tsvgx

This is the revision that shows the regression:
https://tbpl.mozilla.org/?startdate=2014-02-27&enddate=2014-03-01&rev=97072008658c

You can see the value changes while looking at a range of commits:
https://tbpl.mozilla.org/?startdate=2014-02-27&enddate=2014-03-01&jobname=Android%202.2%20Tegra%20mozilla-central%20talos%20remote-tsvgx  (search for revision 97072008658c and then look at the values before/after)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

a better link for viewing the revision:
https://tbpl.mozilla.org/?fromchange=c8e793bdc6eb&tochange=53dbf19f2f8d&jobname=Android%202.2%20Tegra%20mozilla-central%20talos%20remote-tsvgx

Comment 2

[:Gijs (he/him)]

Both Marco and my csets are desktop only... so I don't understand how they'd affect an android testresult. :-\

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

it could be that it affects the svg test and desktop has much more horsepower than android.

Comment 4

[Neil Deakin]

He means that his changes affect files that don't exist on the android version.

My change does, but only affects xul popups which as far as I know aren't used on the Android version either.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

ah, got it.  xul isn't used on android either.  I see a similar change on aurora about 1 day prior to this, I am trying to find changes that landed originally on aurora and then on central.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

highly suspect this could be the python -u change from bug 977298, some retriggers will tell me that answer in a real short time window.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

odd, after a series of retriggers, I keep coming back to the same changeset.  the python change was for android 4.0 only, this is only an issue on android 2.2.  

I did some retriggers and the push to m-c shows an obvious difference:
https://tbpl.mozilla.org/?jobname=talos%20remote-tsvgx&fromchange=58eca03214a6&tochange=5acf5aefe27a

Since these changes are in files that are not on mobile, then something else would need to explain this.  

I have looked at aurora as well:
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=mozilla-aurora%20talos%20remote-tsvgx&fromchange=b921918ab99b&tochange=836391ce81b8

this is interesting as we see a similar change in results, although we are using a different version of talos (talos.<rev>.zip).  In fact our tegras don't have an automatic versioning, nor do we use mozharness, these are more static tools.

Callek: did we update bm-remote, sut_tools or anything else around feb 27th?

Comment 8

[Justin Wood (:Callek)]

I can't find any correlations on the releng side that would have caused this...

No bm-remote or sut_tools updates, we did however have a buildbot reconfig:

https://wiki.mozilla.org/ReleaseEngineering:Maintenance

A mozharness merge:

http://hg.mozilla.org/build/mozharness/graph/c67d51e9ed73

I do see some changes that would have affected talos/mobile, but I don't believe any of them would have been able to cause any measurable regressions.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

thanks Callek for the help.  I am out of ideas here.  Since this is a tegra regression, we need to look at foopy configs instead of mozharness.

I suspect next steps on this bug will be to let it sit idle for 8 weeks or so and then close it as wontfix.  Not sure what else to do.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

this appears to have been fixed for the most part:
http://graphs.mozilla.org/graph.html#tests=[[281,11,29]]&sel=none&displayrange=30&datatype=running

Since we cannot come to a conclusion on this, I am going to mark it as fixed.