Closed Bug 1026798 Opened 6 years ago Closed 6 years ago

1.74e+03 Robopan Android 4.0.4 regression on fx-team (v33) June 12th from push 578bf8d6363c

Categories

(Firefox for Android :: General, defect)

ARM
Android
defect
Not set

Tracking

()

RESOLVED FIXED
Firefox 33

People

(Reporter: jmaher, Assigned: Margaret)

References

Details

(Keywords: perf, regression, Whiteboard: [talos_regression])

here is a graph showing the regression:
http://graphs.mozilla.org/graph.html#tests=[[174,64,29]]&sel=1402512218000,1402685018000&displayrange=7&datatype=running

I did some retriggers to make sure there is no spiders in the tegra farm causing odd numbers:
https://tbpl.mozilla.org/?tree=Fx-Team&fromchange=98d801fe3f5d&tochange=ca66ccfcd177&jobname=Android%204.0%20Panda%20fx-team%20talos%20remote-trobopan

Here is the pushlog:
http://hg.mozilla.org/integration/fx-team/pushloghtml?fromchange=431dd53007b7&tochange=578bf8d6363c

I see in bug 1019318 there is talk about robopan numbers- maybe something didn't get checked in?  I suspect this is easier to fix or explain why the test as it is has become invalid in some way.

btw, I believe this hits the award for the largest number posted from the alert regression email tool, an e+03 is pretty slick and worth some bragging points!
oh, upon further looking, this appears to be fixed a couple of days later- the graph show that- does that make sense to you Margaret?
Flags: needinfo?(margaret.leibovic)
(In reply to Joel Maher (:jmaher) from comment #1)
> oh, upon further looking, this appears to be fixed a couple of days later-
> the graph show that- does that make sense to you Margaret?

Yup, I backed out the original patch in bug 1019735 when I got the email about this regression, then I landed a fixed version.
No longer blocks: 1019318
Status: NEW → RESOLVED
Closed: 6 years ago
Flags: needinfo?(margaret.leibovic)
Resolution: --- → FIXED
(In reply to :Margaret Leibovic from comment #2)
> Yup, I backed out the original patch in bug 1019735 when I got the email
> about this regression, then I landed a fixed version.

\o/

However, it should still be noted that the robopan (or was it robocheck?) numbers are all over the place even without the regression in this bug. Their "normal" values are kinda uniform between 25000 and 100000. Yes, a range of x4 factor.

I talked to kats recently about revisiting this test (he wrote it, and theoretically it does impressive measuring), so maybe it's time to nudge him a bit more? ;)
Flags: needinfo?(bugmail.mozilla)
IIRC we were only discussion robocheck before. I'm not entirely convinced that robopan uses a good algorithm [1] to come up with its number.

[1] http://mxr.mozilla.org/mozilla-central/source/build/mobile/robocop/FennecNativeDriver.java?rev=523e67cf4b9e#129
Flags: needinfo?(bugmail.mozilla)
Assignee: nobody → margaret.leibovic
Target Milestone: --- → Firefox 33
You need to log in before you can comment on or make changes to this bug.