828571 - [tracking] Get unittests running on Android x86 [Emulator]

Status: RESOLVED DUPLICATE of bug 891959

(Testing :: General, defect)

Description

[Bob Clary [:bc] (inactive)]

Tracking bug for getting unittests running on Android x86 emulators

Comment 1

[Dan Minor [:dminor]]

I'll be looking at this for Q2.

Comment 2

[Dan Minor [:dminor]]

I've spent some time running through mochitests. There are several hundred unexpected failures and also another 100 or so tests that will consistently timeout. I also see some process crashes, which I haven't diagnosed yet.

I haven't worked through all of the mochitests yet as it takes awhile to identify and remove timeout causing tests, but I'm going to try the other test harnesses to see what the situation is like there.

Comment 3

[Dan Minor [:dminor]]

Robocop gives an "interesting" error on startup that will require further investigation:

    I/SUTAgentAndroid( 1729): 10.0.2.2 : exec "MOZ_CRASHREPORTER=1,XPCOM_DEBUG_BREAK=stack,MOZ_HIDE_RESULTS_TABLE=1,MOZ_CRASHREPORTER_NO_REPORT=1,NO_EM_RESTART=1,MOZ_PROCESS_LOG=/tmp/tmpJvAusDpidlog,XPCOM_MEM_BLOAT_LOG=/tmp/tmpLRYM2e/runtests_leaks.log" am instrument -w -e deviceroot /mnt/sdcard/tests -e class org.mozilla.fennec_dminor.tests.testAllPagesTab org.mozilla.roboexample.test/org.mozilla.fennec_dminor.FennecInstrumentationTestRunner
    D/AndroidRuntime( 2661):
    D/AndroidRuntime( 2661): >>>>>> AndroidRuntime START com.android.internal.os.RuntimeInit <<<<<<
    D/AndroidRuntime( 2661): CheckJNI is ON
    D/dalvikvm( 2661): Trying to load lib libjavacore.so 0x0
    D/dalvikvm( 2661): Added shared lib libjavacore.so 0x0
    D/dalvikvm( 2661): Trying to load lib libnativehelper.so 0x0
    D/dalvikvm( 2661): Added shared lib libnativehelper.so 0x0
    D/AndroidRuntime( 2661): Calling main entry com.android.commands.am.Am
    D/dalvikvm( 2661): Note: class Landroid/app/ActivityManagerNative; has 156 unimplemented (abstract) methods
    W/ActivityManager( 1197): Permission Denial: startInstrumentation asks to run as user -2 but is calling from user 0; this requires android.permission.INTERACT_ACROSS_USERS_FULL

Comment 4

[Bob Clary [:bc] (inactive)]

dminor, I assume this was with Android 4.2? I've seen this also in Autophone. We are planning on using mozbase's launchFennec which has a fix for this issue. See bug 838844 and bug 862508.

Comment 5

[Dan Minor [:dminor]]

bc, it is under Android 4.2. Thanks for the heads up.

Comment 6

[Dan Minor [:dminor]]

Reftests show a small number of unexpected results (20). Crashtests and JSReftests work fine.

Comment 7

[Dan Minor [:dminor]]

Talos tests run. tsvg, ts_paint work, tp4m runs but has problems parsing the results. trobopan (and other robocop based tests) will not work until the activity manager problem under 4.2 is resolved.

Comment 8

[Dan Minor [:dminor]]

The patch to deviceManagerSUT launchProcess was taken care of by bug 873628, which got the robocop tests running. Fennec crashes intermittently when running robocop and this will require more investigation.

A patch to add an androidx86.json file for mochitests was part of bug 875737, which gives a green run for mochitests in 8 chunks.

Android x86 specific known fails for reftests are being done as part of bug 875801. Some of the fails for regular Android pass on Android x86, so the platform needs to be tracked separately. Crashtests and JSRefTests work without anything special handling.

Mozharness scripts to run mochitest, robocop, reftest, jsreftest and crashtests were done as part of bug 867377 and have been landed in mozharness.

Comment 9

[Geoff Brown [:gbrown]]

We are *still* working on this, but it is tracked in 891959 these days...