757475 - Robocop: testPermissions fails on tegras

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Geoff Brown [:gbrown] (pto, back Jun 10)]

The mochitest-robotium testPermissions test consistently fails, and the failure is not detected: The test appears to pass on tbpl, but something is actually very wrong!

This log is typical:

FIRE PROC: '"MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpdiCugWpidlog,XPCOM_MEM_BLOAT_LOG=/mnt/sdcard/tests/profile/runtests_leaks.log" am instrument -w -e class org.mozilla.fennec.tests.testPermissions org.mozilla.roboexample.test/android.test.InstrumentationTestRunner'
INFO | automation.py | Application pid: 0
0 INFO SimpleTest START
1 INFO TEST-START | testPermissions
2 INFO TEST-PASS | testPermissions | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_geolocation.html should equal http://mochi.test:8888/tests/robocop/robocop_geolocation.html
3 INFO TEST-PASS | testPermissions | checkbox count - only one checkbox visible
4 INFO TEST-PASS | testPermissions | checkbox checked - checkbox is checked
5 INFO TEST-PASS | testPermissions | checkbox not checked - checkbox is not checked
INFO | automation.py | Application ran for: 0:05:48.386831
INFO | automation.py | Reading PID log: /tmp/tmpdiCugWpidlog
getting files in '/mnt/sdcard/tests/profile/minidumps/'
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

Note:
 - runs for over 5 minutes
 - only the first few assertions are reported; the test contains many more!

Comment 1

[Geoff Brown [:gbrown] (pto, back Jun 10)]

Unfortunately, this test runs perfectly when I run it locally:

0 INFO SimpleTest START
1 INFO TEST-START | testPermissions
2 INFO TEST-PASS | testPermissions | Awesomebar URL typed properly - http://mochi.test:8888/tests/robocop/robocop_geolocation.html should equal http://mochi.test:8888/tests/robocop/robocop_geolocation.html
3 INFO TEST-PASS | testPermissions | checkbox count - only one checkbox visible
4 INFO TEST-PASS | testPermissions | checkbox checked - checkbox is checked
5 INFO TEST-PASS | testPermissions | checkbox not checked - checkbox is not checked
6 INFO TEST-PASS | testPermissions | checking page background is green - Color rgba(0,130,0,255)  close enough to expected rgb(0,128,0)
7 INFO TEST-PASS | testPermissions | checkbox checked - checkbox is checked
8 INFO TEST-PASS | testPermissions | checking page background is green - Color rgba(0,130,0,255)  close enough to expected rgb(0,128,0)
9 INFO TEST-PASS | testPermissions | checking page background is green - Color rgba(0,130,0,255)  close enough to expected rgb(0,128,0)
10 INFO TEST-END | testPermissions | finished in 46425ms
11 INFO TEST-START | Shutdown
12 INFO Passed: 8
13 INFO Failed: 0
14 INFO Todo: 0
15 INFO SimpleTest FINISHED
INFO | automation.py | Application ran for: 0:01:05.438709
INFO | automation.py | Reading PID log: /tmp/tmpet3GmZpidlog
WARNING | automationutils.processLeakLog() | refcount logging is off, so leaks can't be detected!

Comment 2

[:Margaret Leibovic]

I was hoping this wouldn't be a problem, but it looks like the geolocation API is having trouble on the tinderbox devices. This test depends on a successful geolocation callback to turn the page green, but it looks like it's timing out waiting for that. I'm not sure how to fix this.

Doug, do we have other geolocation API tests in the tree that run on our Android test devices? Do they get actual position data from the devices?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I believe the mochitest geolocation tests are not run on android.

Comment 4

[Doug Turner (:dougt)]

On android, no.  On android, we use the system geolocation service and do not have the testable wifi/ip network based gelocation provider.

On all other platforms, we use a mock location provider.

Comment 5

[Geoff Brown [:gbrown] (pto, back Jun 10)]

Perhaps we should just disable this test then (comment out in robocop.ini with a pointer back to this bug)...any better ideas?

Comment 6

[Geoff Brown [:gbrown] (pto, back Jun 10)]

Attachment: disable testPermissions

Comment 7

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/fa66afa7f6c7

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/fa66afa7f6c7