Closed Bug 978878 Opened 12 years ago Closed 12 years ago

3-4% regression on tsvgx for android on mozilla-central

Categories

(Testing :: Talos, defect)

Product:
Testing
Component:
Talos
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jmaher, Unassigned)

Details

(Keywords: perf, regression, Whiteboard: [talos_regression])

and talos alerts keep finding more stuff: https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/_0YELL43jfE here is a definition of the tests: https://wiki.mozilla.org/Buildbot/Talos/Tests#tsvg.2C_tsvgx This is the revision that shows the regression: https://tbpl.mozilla.org/?startdate=2014-02-27&enddate=2014-03-01&rev=97072008658c You can see the value changes while looking at a range of commits: https://tbpl.mozilla.org/?startdate=2014-02-27&enddate=2014-03-01&jobname=Android%202.2%20Tegra%20mozilla-central%20talos%20remote-tsvgx (search for revision 97072008658c and then look at the values before/after)
Both Marco and my csets are desktop only... so I don't understand how they'd affect an android testresult. :-\
it could be that it affects the svg test and desktop has much more horsepower than android.
He means that his changes affect files that don't exist on the android version. My change does, but only affects xul popups which as far as I know aren't used on the Android version either.
ah, got it. xul isn't used on android either. I see a similar change on aurora about 1 day prior to this, I am trying to find changes that landed originally on aurora and then on central.
highly suspect this could be the python -u change from bug 977298, some retriggers will tell me that answer in a real short time window.
odd, after a series of retriggers, I keep coming back to the same changeset. the python change was for android 4.0 only, this is only an issue on android 2.2. I did some retriggers and the push to m-c shows an obvious difference: https://tbpl.mozilla.org/?jobname=talos%20remote-tsvgx&fromchange=58eca03214a6&tochange=5acf5aefe27a Since these changes are in files that are not on mobile, then something else would need to explain this. I have looked at aurora as well: https://tbpl.mozilla.org/?tree=Mozilla-Aurora&jobname=mozilla-aurora%20talos%20remote-tsvgx&fromchange=b921918ab99b&tochange=836391ce81b8 this is interesting as we see a similar change in results, although we are using a different version of talos (talos.<rev>.zip). In fact our tegras don't have an automatic versioning, nor do we use mozharness, these are more static tools. Callek: did we update bm-remote, sut_tools or anything else around feb 27th?
Flags: needinfo?(bugspam.Callek)
I can't find any correlations on the releng side that would have caused this... No bm-remote or sut_tools updates, we did however have a buildbot reconfig: https://wiki.mozilla.org/ReleaseEngineering:Maintenance A mozharness merge: http://hg.mozilla.org/build/mozharness/graph/c67d51e9ed73 I do see some changes that would have affected talos/mobile, but I don't believe any of them would have been able to cause any measurable regressions.
Flags: needinfo?(bugspam.Callek)
thanks Callek for the help. I am out of ideas here. Since this is a tegra regression, we need to look at foopy configs instead of mozharness. I suspect next steps on this bug will be to let it sit idle for 8 weeks or so and then close it as wontfix. Not sure what else to do.
this appears to have been fixed for the most part: http://graphs.mozilla.org/graph.html#tests=[[281,11,29]]&sel=none&displayrange=30&datatype=running Since we cannot come to a conclusion on this, I am going to mark it as fixed.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.