756377 - Zooming on nytimes has regressed since the 4th of May

Status: RESOLVED INVALID

(Firefox for Android Graveyard :: General, defect)

Description

[Jeff Muizelaar [:jrmuizel]]

If you take a look here: http://wrla.ch/eideticker/dashboard/#/nytimes-zooming/checkerboard you can see that we've gotten noticeably worse at zooming out on nytimes.com. Unfortunately, there's not much data so we have a pretty big window.

Comment 1

[Jeff Muizelaar [:jrmuizel]]

One possible cause is bug 750598.

Comment 2

[William Lachance (:wlach)]

A few things to bear in mind:

* The latency characteristics of the input events on the test may have changed slightly, because we're now using a different method to communicate them to the device (sut-over-tcp vs. adb-over-usb before).
* The regression, if there is one, is actually fairly minor.

That said, this is probably worth investigating, especially if there has been work that might have caused a regression.

Comment 3

[William Lachance (:wlach)]

BTW, I would be happy to help track this down next week if it would be helpful.

See also: 

Bug 756428 -  Would be useful to have a mozregression-like tool for tracking down regressions that Eideticker finds

Comment 4

[Mark Finkle (:mfinkle) (use needinfo?)]

William - Can you run some builds betwen 5/4 and 5/15? Eideticker is missing data in that range.

Comment 5

[William Lachance (:wlach)]

Updated the regression finding tool a bit to support downloading fennec builds, and used it to try and track this down. To speed things up, I used telemetry (aka eideticker-with-less-eideticker: http://wrla.ch/blog/2012/04/eideticker-with-less-eideticker/). Here's a csv version of what I found.

build,result1,result2,result3,average
nightly-2012-5-1.apk,1.261635,1.852615,1.261635,1.45862833333
nightly-2012-5-2.apk,1.852615,3.982601,2.512977,2.782731
nightly-2012-5-3.apk,4.760391,3.982599,4.760395,4.50112833333
nightly-2012-5-4.apk,1.261635,1.852615,1.852615,1.65562166667
nightly-2012-5-5.apk,2.512978,1.261635,3.228955,2.33452266667
nightly-2012-5-6.apk,2.512978,1.261635,3.228955,2.33452266667
nightly-2012-5-7.apk,1.852615,3.228955,1.261635,2.11440166667
nightly-2012-5-8.apk,2.512978,3.982601,5.545099,4.01355933333
nightly-2012-5-9.apk,5.545096,5.545096,3.228954,4.77304866667
nightly-2012-5-10.apk,3.228954,4.760395,5.545099,4.51148266667
nightly-2012-5-11.apk,3.982601,3.982601,4.760395,4.24186566667
nightly-2012-5-12.apk,5.545099,3.982601,5.545096,5.02426533333
nightly-2012-5-13.apk,3.228954,5.545098,4.760395,4.51148233333
nightly-2012-5-14.apk,5.545099,3.228955,3.982601,4.25221833333
nightly-2012-5-15.apk,3.982605,4.760395,3.982602,4.24186733333

It looks like things got worse with the nightly on the 3rd, then slightly better, then worse again on the 8th. Note that numbers this low represent only a handful of checkerboarded frames (less than a second's worth), so the regression (if there is one) is fairly subtle.

Comment 6

[Johnathan Nightingale [:johnath]]

Minus for now, please re-nom if you think it deserves to block

Comment 7

[William Lachance (:wlach)]

So I fixed bug 749472 to try and get the numbers more numerically stable. Here's the results with that patch enabled:

build,result1,result2,result3,average
nightly-2012-5-1.apk,4.760395,1.261635,2.512978,2.84500266667
nightly-2012-5-2.apk,1.852615,3.982605,3.228956,3.021392
nightly-2012-5-3.apk,2.512978,1.852615,1.261635,1.87574266667
nightly-2012-5-4.apk,1.852615,3.22896,0.76159,1.94772166667
nightly-2012-5-5.apk,4.760395,4.760395,3.982603,4.501131
nightly-2012-5-6.apk,4.760395,4.400131,3.577782,4.24610266667
nightly-2012-5-7.apk,5.545101,5.545101,5.545101,5.545101
nightly-2012-5-8.apk,5.545101,4.760395,5.5326,5.27936533333
nightly-2012-5-9.apk,5.545101,5.545101,5.545101,5.545101
nightly-2012-5-10.apk,4.760395,5.545101,5.5451,5.283532
nightly-2012-5-11.apk,5.5451,5.545101,3.22896,4.77305366667
nightly-2012-5-12.apk,5.545101,5.545101,5.545101,5.545101
nightly-2012-5-13.apk,4.760395,3.982605,3.22896,3.99065333333
nightly-2012-5-14.apk,3.22896,4.760395,3.22896,3.73943833333
nightly-2012-5-15.apk,5.545101,3.982605,5.545101,5.024269

You'll note that these results don't correlate with the pattern I saw before, but that they continue to bounce all over the place. No idea exactly what's going on here, but I think we can rule out a major regression (remember again that we're talking about a few sections of the test checkerboarding over a small fraction of a second).

I would recommend resolving this bug as invalid.

Comment 8

[Kevin Brosnan [Ex-Mozilla]]

Clearing regression window wanted and closing per comment 7.