Closed
Bug 777357
Opened 12 years ago
Closed 9 years ago
Apparent regression in CNN benchmark between 2012-07-22 and 2012-07-23
Categories
(Firefox for Android Graveyard :: Toolbar, defect, P5)
Tracking
(fennec+)
RESOLVED
DUPLICATE
of bug 785069
Firefox 17
Tracking | Status | |
---|---|---|
fennec | + | --- |
People
(Reporter: wlach, Assigned: snorp)
References
Details
(Keywords: perf, regression)
Attachments
(1 file)
http://wrla.ch/eideticker/dashboard/#/cnn/checkerboard seems to indicate a regression between 2012-07-22 and 2012-07-23. I will run a quick manual test to verify this.
Updated•12 years ago
|
tracking-fennec: --- → ?
Reporter | ||
Comment 1•12 years ago
|
||
Yeah, there definitely seems to be a regression here: http://people.mozilla.com/~wlachance/cnn-benchmark/metric.html?data=metric-test-1343419943.79.json#/checkerboard The build on the 24th corresponds to: 100108:3613cbdc3481 The build on the 23rd corresponds to: 100084:82b6c5885345 Looking at the changelog, nothing is particularly jumping out at me. I guess we'll have to try profiling against a buncha tinderbox builds.
Reporter | ||
Comment 2•12 years ago
|
||
Here's some results from running internal telemetry against CNN.com for the regression range I noted. Not seeing where the issue might be.
Assignee: nobody → wlachance
Reporter | ||
Comment 3•12 years ago
|
||
I'm pretty stumped about where to take this next. If you look at the dashboard, the overall trend does seem to be "up", but it's not a strong enough difference to easily pinpoint the difference. I wonder if we need to try a harder test here?
Updated•12 years ago
|
Assignee: wlachance → snorp
tracking-fennec: ? → 17+
Updated•12 years ago
|
Keywords: regression,
regressionwindow-wanted
Comment 4•12 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/9b6f5ce1065a
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 17
Comment 5•12 years ago
|
||
Typoed bug number in 9b6f5ce1065a, seems to be bug 777537.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Somewhere between : https://tbpl.mozilla.org/?rev=ef20925bc2a5 https://tbpl.mozilla.org/?rev=a26e751bfb54 https://tbpl.mozilla.org/?rev=d78729026fb9
Oh oops. need to go back a bit further. My guess is : http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/07/2012-07-22-03-05-55-mozilla-central-android/ will not show the issue. http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/07/2012-07-23-03-06-06-mozilla-central-android/ probably will show the issue. https://tbpl.mozilla.org/?rev=d78729026fb9&jobname=Android%20Debug%20mozilla-central%20build Is there an easy way to test builds : using revision: mozilla-central/57499abe6479 using revision: mozilla-central/19fe7ecd0af8 My guess is something in : https://tbpl.mozilla.org/?rev=19fe7ecd0af8&jobname=Android%20Debug%20mozilla-central%20build But I am not sure. I can't set up Eideticker on this machine because I don't have HDMI in. Pinging :wlach for further possible investigation?
Reporter | ||
Comment 8•12 years ago
|
||
(In reply to Naoki Hirata :nhirata from comment #7) > Oh oops. need to go back a bit further. > > My guess is : > http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/07/2012-07-22-03- > 05-55-mozilla-central-android/ > will not show the issue. > > http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/07/2012-07-23-03- > 06-06-mozilla-central-android/ > probably will show the issue. > > https://tbpl.mozilla.org/?rev=d78729026fb9&jobname=Android%20Debug%20mozilla- > central%20build > > Is there an easy way to test builds : > using revision: mozilla-central/57499abe6479 > using revision: mozilla-central/19fe7ecd0af8 > > My guess is something in : > https://tbpl.mozilla.org/?rev=19fe7ecd0af8&jobname=Android%20Debug%20mozilla- > central%20build > > But I am not sure. I can't set up Eideticker on this machine because I > don't have HDMI in. Pinging :wlach for further possible investigation? This is almost certainly a duplicate of bug 785069. It looks like the regression is much more obvious with the robocop benchmark in that bug, so that's probably what I'd use to try and reproduce it.
Updated•12 years ago
|
Keywords: regressionwindow-wanted
Comment 9•12 years ago
|
||
There is still a need for a regression range from one (or both) of the two bugs.
Keywords: regressionwindow-wanted
Comment 10•12 years ago
|
||
Is this bug even still an issue? I don't think QA has the Eideticker setup or resources to look into a regression range here. William, can you take a stab at doing so if it deems necessary still?
Reporter | ||
Comment 11•12 years ago
|
||
(In reply to Aaron Train [:aaronmt] from comment #10) > Is this bug even still an issue? I don't think QA has the Eideticker setup > or resources to look into a regression range here. William, can you take a > stab at doing so if it deems necessary still? As I mention above, this is almost certainly a duplicate of bug 785069, which both has a more obvious regression and can be reproduced without eideticker. It looks like the ball is in David Zbarsky's court there, I'll take the liberty of doing a ping check there. If we find the cause of the regression and a fix in that bug, I'd be happy to verify the improvement on eideticker using my setup.
David Zbarsky was an intern who is back in school. I am not sure if we will get a response from him about this bug.
Flags: needinfo?(wlachance)
Reporter | ||
Comment 13•12 years ago
|
||
(In reply to Naoki Hirata :nhirata from comment #12) > David Zbarsky was an intern who is back in school. I am not sure if we will > get a response from him about this bug. Regardless, it looks like bug 785069 is the place to look for regressions. Let's see what happens there, it looks like :kbrosnan has been doing some good work tracking down the offending changeset.
Flags: needinfo?(wlachance)
Updated•12 years ago
|
tracking-fennec: 17+ → +
Comment 14•11 years ago
|
||
This slipped down my list of things to look at. This test does not seem to be running currently http://graphs.mozilla.org/graph.html#tests=[[204,63,20]]&sel=none&displayrange=365&datatype=running Pulling regression window for the time being.
Keywords: regressionwindow-wanted
Comment 15•11 years ago
|
||
Was that for tcheck2? I'm not sure what's wrong with the link in Comment 14, but this one shows current results for tcheck2: http://graphs.mozilla.org/graph.html#tests=[[201,63,20]]&sel=none&displayrange=365&datatype=running
Comment 17•9 years ago
|
||
Per comment 11, comment 13.
Status: REOPENED → RESOLVED
Closed: 12 years ago → 9 years ago
Resolution: --- → DUPLICATE
Updated•3 years ago
|
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•