Closed Bug 851861 Opened 10 years ago Closed 8 years ago

Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad, testFlingCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)

Categories

(Firefox for Android Graveyard :: Toolbar, defect)

Product:
Firefox for Android Graveyard
Component:
Toolbar
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox24 disabled, firefox25 disabled, firefox26 disabled, firefox35 wontfix, firefox36 fixed, firefox37+ fixed, firefox38 fixed, firefox-esr31 wontfix)

Status:
RESOLVED FIXED
Milestone:
Firefox 38
Tracking Flags:
Tracking Status
firefox24 --- disabled
firefox25 --- disabled
firefox26 --- disabled
firefox35 --- wontfix
firefox36 --- fixed
firefox37 + fixed
firefox38 --- fixed
firefox-esr31 --- wontfix

People

(Reporter: philor, Assigned: capella)

References

Details

(Keywords: intermittent-failure)

Attachments

(5 files, 1 obsolete file)

screenshot of Robocop Boxes
31.12 KB, image/png
Details
screenshot captured in recent failures
29.20 KB, image/jpeg
Details
bug851861.diff
10.32 KB, patch
kats
: review+
mfinkle
: approval-mozilla-aurora-
mfinkle
: approval-mozilla-beta-
Details | Diff | Splinter Review
bug851861_v1_backout.diff
9.41 KB, patch
mfinkle
: review+
Details | Diff | Splinter Review
bug851861_v2.diff
7.23 KB, patch
mfinkle
: review+
Sylvestre
: approval-mozilla-aurora+
Sylvestre
: approval-mozilla-beta+
Details | Diff | Splinter Review 
https://tbpl.mozilla.org/php/getParsedLog.php?id=20731093&tree=Mozilla-Inbound
Android 4.0 Panda mozilla-inbound opt test robocop-1 on 2013-03-16 17:35:20 PDT for push 3648f5fa5787
slave: panda-0846

4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:55:54.414 I/Robocop ( 3716): 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:55:54.421 I/Robocop ( 3716): junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:55:54.429 I/Robocop ( 3716): 5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:55:55.656 I/TestRunner( 3716): junit.framework.AssertionFailedError: 5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)

https://tbpl.mozilla.org/php/getParsedLog.php?id=20736408&tree=Mozilla-Inbound
Android 4.0 Panda mozilla-inbound opt test robocop-1 on 2013-03-16 21:26:17 PDT for push 730fbb35edd5
slave: panda-0588

4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 21:42:38.351 I/Robocop ( 5744): 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 21:42:38.367 I/Robocop ( 5744): junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 21:42:38.367 I/Robocop ( 5744): 5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 21:42:39.625 I/TestRunner( 5744): junit.framework.AssertionFailedError: 5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)

https://tbpl.mozilla.org/php/getParsedLog.php?id=20730880&tree=Mozilla-Inbound
Android 4.0 Panda mozilla-inbound opt test robocop-1 on 2013-03-16 17:29:16 PDT for push 29fdd1949b09
slave: panda-0657

4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:51:24.796 I/Robocop ( 3443): 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:51:24.804 I/Robocop ( 3443): junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:51:24.804 I/Robocop ( 3443): 5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:51:26.078 I/TestRunner( 3443): junit.framework.AssertionFailedError: 5 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:52:31.218 I/Robocop ( 3684): 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:52:31.226 I/Robocop ( 3684): junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:52:31.226 I/Robocop ( 3684): 5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
03-16 17:52:32.515 I/TestRunner( 3684): junit.framework.AssertionFailedError: 5 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testOverscroll | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
https://tbpl.mozilla.org/php/getParsedLog.php?id=20737177&tree=Mozilla-Inbound
Summary: Intermittent testOverscroll, testPanCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0) → Intermittent testOverscroll, testPanCorrectness, testAxisLocking | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
https://tbpl.mozilla.org/php/getParsedLog.php?id=20751244&tree=Mozilla-Inbound
Summary: Intermittent testOverscroll, testPanCorrectness, testAxisLocking | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0) → Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
https://tbpl.mozilla.org/php/getParsedLog.php?id=20782611&tree=Mozilla-Inbound
Summary: Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0) → Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad, testFlingCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
The first instance of this I see is 

https://tbpl.mozilla.org/php/getParsedLog.php?id=20716358&tree=Mozilla-Inbound&full=1

from https://hg.mozilla.org/integration/mozilla-inbound/rev/60261728879e, Fri Mar 15 21:35:45 2013 PDT. 

This must be a regression from something landed on m-i on Mar 15, but there was so much bustage that day, it won't be easy to identify.
(In reply to Geoff Brown [:gbrown] from comment #9)
> This must be a regression from something landed on m-i on Mar 15, but there
> was so much bustage that day, it won't be easy to identify.

Not easy, but not impossible!

Blame lands on https://hg.mozilla.org/integration/mozilla-inbound/rev/a71c3f178d91.

Compare:

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=Android&rev=a71c3f178d91
and
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=Android&rev=921e9f7f8924
Blocks: 850690
I'll take a look at this - please let's not back this out unless we absolutely have to, this commit fixes a real bug that can be triggered regardless of the dynamic toolbar stuff (if you trigger the keyboard or rotate during startup, you can trigger it)

I expect these failures indicate there's still a way for compositor and Gecko window size to mismatch, but I'll need to look closer. How urgent is this, can we take this failure for a while? (I'm at a work week now and have a week of holiday after)
Adding needinfo? for comment #21.
Flags: needinfo?(philringnalda)
We could disable the tests for a couple of weeks, but there are at least 5 of them: testLoad, testOverscroll, testAxisLocking, testPanCorrectness, and testFlingCorrectness.
The way you get away with that is to take the bug, and then keep your head down and keep quiet, not broadcast "I plan on leaving this broken" as widely as possible.
Assignee: nobody → chrislord.net
Flags: needinfo?(philringnalda)
Depends on: 852526
It looks like this wasn't fixed by bug 852526, so I guess there's some race condition somewhere with setting the dynamic toolbar pref during robocop test startup(?)
(This is one of our more frequent oranges the last week)
(In reply to Chris Lord [:cwiiis] from comment #49)
> It looks like this wasn't fixed by bug 852526

I think it was fixed: Notice that Comment 48 is a slightly different error (rgba(0,101,33,255) vs rgba(0,0,0,255) makes it bug 836815) and that is the only failure since the landing in bug 852526.
Marking as fixed, wrt comment #51 - please reopen if not :)
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
(In reply to Chris Lord [:cwiiis] from comment #52)
> Marking as fixed, wrt comment #51 - please reopen if not :)

ok
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I get this failure approximately 1 out of 10 times I run theOverscroll locally on an HTC Desire (Android 2.2). From what I see the block from page load is returned to fast and the "boxes" are not displayed yet when checking for the color. I also see a slowness sometimes to render content for the first page loaded after the test starts. Maybe we should just load a Robocop Blank page before this one.
After doing some investigations on this adding a loadUrl of the Robocop blank page first and then Robocop boxes seems to solve the failure of the first check. Unfortunately at least on my Desire (Android 2.2) the dragScroll from testPanCorrectness and testAxisLock is always 6 pixels short although it's done over 100 pixels. Adding 6 pixels to the drag scrolls correctly but i still get a fail for the color checks from (100,100) to (200,200) - the last part of the checks - because for some reason it checks the color at 0,0 and gets(0,0,0) instead of (32.200.32).
Attached image Screenshot htc desire Z (obsolete) — 
Looking into this locally on an HTC Desire Z (Android 2.3.3) the algorithm for the boxes seems off. The first row seems wider then the second - please see the attached screenshot. I will investigate further.
The difference is more visible in this screenshot
Attachment #777007 - Attachment is obsolete: true
Printing every pixel in Robocop boxes it seems that the first box with rgb(0,0,0,255) is 149 pixels high and 103 wide. The rest of the boxes continue similarly with widths and heights that are off from the defined 100 px high and 100 px wide.
(In reply to Adrian Tamas from comment #130)
> Printing every pixel in Robocop boxes it seems that the first box with
> rgb(0,0,0,255) is 149 pixels high and 103 wide. 

That does not seem consistent with the screen shot: each box in the screenshot looks roughly square to me.

Another technique that may be instructive is to collect the file /mnt/sdcard/tests/pixels.map and analyze that.
It may be a visual illusion because of the boxes but the second row looks narrower to me and especially on a device.
:cwiiis -- Do you know what's going wrong here? Are you making any progress? This is one of the most frequent test failures on tbpl currently.
Flags: needinfo?(chrislord.net)
(In reply to Geoff Brown [:gbrown] from comment #159)
> :cwiiis -- Do you know what's going wrong here? Are you making any progress?
> This is one of the most frequent test failures on tbpl currently.

sorry, I forgot this was assigned to me... I haven't made any progress and I only have vague ideas as to what this could be.

I could use some help here, I don't know if I really have time for this. I could certainly help guide someone through root-causing it.
Flags: needinfo?(chrislord.net)