Closed Bug 784242 Opened 8 years ago Closed 8 years ago

Robocop: PixelTests failures: Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)

Categories

(Testing :: General, defect)

ARM
Android
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla22

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(3 files)

In local tests on the Galaxy Nexus, several PixelTest-derived tests fail with:

Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)

The roboboxes test page loads and looks okay. On tegras and other devices, the test seems to run fine (except for the panda boards -- bug 783639...but that's probably unrelated).
0 INFO SimpleTest START
1 INFO TEST-START | testAxisLocking
2 INFO TEST-PASS | testAxisLocking | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_boxes.html should equal http://mochi.test:8888/tests/robocop/robocop_boxes.html
3 INFO TEST-PASS | testAxisLocking | Pixel at 0, 0 - Color rgba(0,0,0,255)  close enough to expected rgb(0,0,0)
4 INFO TEST-UNEXPECTED-FAIL | testAxisLocking | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
Exception caught during test!
junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testAxisLocking | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
	at junit.framework.Assert.fail(Assert.java:47)
	at org.mozilla.fennec_mozdev.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:107)
	at org.mozilla.fennec_mozdev.FennecMochitestAssert.ok(FennecMochitestAssert.java:136)
	at org.mozilla.fennec_mozdev.FennecMochitestAssert.ispixel(FennecMochitestAssert.java:166)
	at org.mozilla.fennec_mozdev.tests.PixelTest.checkScrollWithBoxes(PixelTest.java:80)
	at org.mozilla.fennec_mozdev.tests.PixelTest.loadAndVerifyBoxes(PixelTest.java:94)
	at org.mozilla.fennec_mozdev.tests.testAxisLocking.testAxisLocking(testAxisLocking.java:29)
	at java.lang.reflect.Method.invokeNative(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:511)
	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
	at org.mozilla.fennec_mozdev.tests.BaseTest.runTest(BaseTest.java:94)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:537)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1551)
5 INFO TEST-UNEXPECTED-FAIL | testAxisLocking | Exception caught - junit.framework.AssertionFailedError: 4 INFO TEST-UNEXPECTED-FAIL | testAxisLocking | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
6 INFO TEST-END | testAxisLocking | finished in 12958ms
This is not specific to the Galaxy Nexus. It fails just the same on the Galaxy S...but obviously not on the tegras.
Summary: Robocop: PixelTests failures on Galaxy Nexus: Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0) → Robocop: PixelTests failures: Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
Works fine on the pandaboard now:

0 INFO SimpleTest START
1 INFO TEST-START | testAxisLocking
2 INFO TEST-PASS | testAxisLocking | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_boxes.html should 
equal http://mochi.test:8888/tests/robocop/robocop_boxes.html
3 INFO TEST-PASS | testAxisLocking | Pixel at 0, 0 - Color rgba(0,0,0,255)  close enough to expected rgb(0,0,0)
4 INFO TEST-PASS | testAxisLocking | Pixel at 100, 0 - Color rgba(33,101,0,255)  close enough to expected rgb(32,100,0)
5 INFO TEST-PASS | testAxisLocking | Pixel at 0, 100 - Color rgba(0,101,33,255)  close enough to expected rgb(0,100,32)
6 INFO TEST-PASS | testAxisLocking | Pixel at 100, 100 - Color rgba(33,203,33,255)  close enough to expected rgb(32,200,32)
7 INFO TEST-PASS | testAxisLocking | Pixel at 0, 0 - Color rgba(0,101,33,255)  close enough to expected rgb(0,100,32)
8 INFO TEST-PASS | testAxisLocking | Pixel at 100, 0 - Color rgba(33,203,33,255)  close enough to expected rgb(32,200,32)
9 INFO TEST-PASS | testAxisLocking | Pixel at 0, 100 - Color rgba(0,203,66,255)  close enough to expected rgb(0,200,64)
10 INFO TEST-PASS | testAxisLocking | Pixel at 100, 100 - Color rgba(33,44,66,255)  close enough to expected rgb(32,44,64)
11 INFO TEST-PASS | testAxisLocking | Pixel at 99, 0 indicates no horizontal scroll - Color rgba(0,101,33,255)  close enough to expe
cted rgb(0,100,32)
12 INFO TEST-PASS | testAxisLocking | Pixel at 0, 0 - Color rgba(33,44,66,255)  close enough to expected rgb(32,44,64)
13 INFO TEST-PASS | testAxisLocking | Pixel at 100, 0 - Color rgba(66,146,66,255)  close enough to expected rgb(64,144,64)
14 INFO TEST-PASS | testAxisLocking | Pixel at 0, 100 - Color rgba(33,146,99,255)  close enough to expected rgb(32,144,96)
15 INFO TEST-PASS | testAxisLocking | Pixel at 100, 100 - Color rgba(66,247,99,255)  close enough to expected rgb(64,244,96)
16 INFO TEST-END | testAxisLocking | finished in 29987ms
17 INFO TEST-START | Shutdown
18 INFO Passed: 14
19 INFO Failed: 0
20 INFO Todo: 0
21 INFO SimpleTest FINISHED
I recently re-tested. This is still a persistent problem on the Galaxy S and Galaxy Nexus; it still works fine on the Tegra and Pandaboard.
Blocks: 824748
It fails on Samsung Galaxy Note also
Hardware: x86 → ARM
hexdump -v | head / tail shows:

0000000 00ff 8492 00ff 8492 00ff 8492 00ff 8492
0000010 00ff 8492 00ff 8492 00ff 8492 00ff 8492
0000020 00ff 8492 00ff 8492 00ff 8492 00ff 8492
...
01436c0 63ff 002c 63ff 002c 63ff 002c 63ff 002c
01436d0 63ff 002c 63ff 002c 63ff 002c 63ff 002c
01436e0 63ff 002c 63ff 002c 63ff 002c 63ff 002c
01436f0 63ff 002c 63ff 002c 63ff 002c 63ff 002c
0143700
The Galaxy S screen is 480(w)x800(h).

Debugging indicates that the getSurfaceView returns a view with dimensions 480(w)x690(h). This is consistent with the pixels.map file size: 480x690x4=1324800.

Debugging shows that each "box" is 150x150, as reported by PaintedSurface.getPixelAt(). This is consistent with the screenshot: 3 boxes + a little bit across; about 4.5 boxes high.
The zoomFactor seems to be 1.5 on phones; 1.0 on tablets.

We could retrieve the viewport metrics in the test and scale our expectations based on the zoom factor....or is there a way to force the zoom to 1.0? kats -- any thoughts here?
Flags: needinfo?(bugmail.mozilla)
The robocop boxes file specifies an initial scale of 1.0. I think what's happening here is that the device dpi scaling [1] is kicking in. So it's not a phones vs tablets thing, it's a display density thing. If we can set a pref in the test environment, then setting browser.viewport.scaleRatio to 100 should fix this. Bug 737090 was wontfix'd or we might have been able to use that instead. If we can't set a pref, then yes, we can update the test code to also do the scaleRatio calculation and adjust accordingly.

[1] https://hg.mozilla.org/mozilla-central/file/ee4879719f78/mobile/android/chrome/content/browser.js#l5550
Flags: needinfo?(bugmail.mozilla)
That pref is just what I was looking for - thanks!

It fixes this test on the Galaxy S and does not regress any tests on tegras or pandas: https://tbpl.mozilla.org/?tree=Try&rev=5fb97f9dbbac
Attachment #722364 - Flags: review?(bugmail.mozilla)
Attachment #722364 - Flags: review?(bugmail.mozilla) → review+
https://hg.mozilla.org/mozilla-central/rev/563213431c9f
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla22
You need to log in before you can comment on or make changes to this bug.