940068 - [Meta] Some "Android 4.0 debug" tests fail

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Geoff Brown [:gbrown]]

+++ This bug was initially created as a clone of Bug #866795 +++

We have tests running on Cedar for Android 4.0 debug builds and there is interest in running those tests on all trunk trees, but several tests fail frequently.

This meta bug tracks our efforts to turn this green:

https://tbpl.mozilla.org/?tree=Cedar&showall=1&jobname=android.*debug

Comment 1

[Geoff Brown [:gbrown]]

Bug 682129 is important here because most of the remaining failures are assertion failures, reported in the main log only as "Assertion count N is greater than expected range m-n assertions." The specific assertions have been reported in logcat, but the logcat buffer has been over-written before being dumped to the main log.

Comment 2

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #1)
> Bug 682129 is important here because most of the remaining failures are
> assertion failures, reported in the main log only as "Assertion count N is
> greater than expected range m-n assertions." The specific assertions have
> been reported in logcat, but the logcat buffer has been over-written before
> being dumped to the main log.

Complete logcats are available now. See http://gbrownmozilla.wordpress.com/2014/02/12/complete-logcats-for-android-tests/.

Comment 3

[Geoff Brown [:gbrown]]

Android Debug test status:
 - mochitests (M1-M8) and js-reftests (J1-J3) are running on trunk
 - mochitest-gl has a consistent crash - bug 1008433
 - robocop has 2 consistent crashes - bug 1008423 and bug 1008426
 - crashtests have a few remaining assertion mismatches - bug 962676
 - we have not yet tried to run plain-reftests, xpcshell, or cppunit tests - bug 1008373
 - we would like to report assertions better - bug 1000995

Comment 4

[Geoff Brown [:gbrown]]

Android 4.0 Debug xpcshell tests now run on trunk trees.

Comment 5

[Geoff Brown [:gbrown]]

We hope to start running Android Debug tests soon on the 4.3 emulator and retire the corresponding Android 4.0 Debug tests. Further efforts to resolve Android debug test failures might be better applied to Android 4.3, bug 1140454.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

I think we are good now.