1146574 - Add missing Android 4.3 test suites: cppunittests, jittests, instrumentation

Status: RESOLVED WONTFIX

(Testing :: General, defect)

Description

[Geoff Brown [:gbrown]]

I'm hoping we can start running some Android 4.3 Opt emulator tests soon, and gradually removing corresponding jobs from Android 4.0 Opt. The 4.3 test suite configs are mirrors of their closest cousins, the Android 2.3 tests: xpcshell, mochitest, reftest, crashtest, jsreftest, robocop. 

There are some additional Android 4.0 test suites which I should probably try to get running on 4.3:

cppunittests - running on all trees in Android 4.0 Opt (but not 2.3, not x86) and running in Android 4.0 Debug only on Cedar
jittests - Opt and Debug running only on Cedar
instrumentation - Opt running only on Cedar

Comment 1

[Dan Minor [:dminor]]

The jittests on Android have never been green since they've been enabled on Cedar (Dec 2013, maybe?). I don't think it's worth spending your time getting them running on the emulator until the js team has time available to investigate and fix the failures. Bug 1098508 has some dependent bugs for the failures from the last time I went through them.

AFAIK the jittests are also enabled on Try for Android 4.0, but hidden by default.

Comment 2

[Geoff Brown [:gbrown]]

Attachment: update mozharness config for cppunit tests on Android 4.3

Comment 3

[Geoff Brown [:gbrown]]

Attachment: update in-tree config and manifest for cppunit tests on android 4.3

Comment 4

[Geoff Brown [:gbrown]]

I can run cppunit tests successfully on 4.3 with these changes:

http://ftp.mozilla.org/pub/mozilla.org/mobile/try-builds/[e-mail]-4aba3befeb42/try-android-api-9/try_ubuntu64_vm_mobile_test-xpcshell-3-bm116-tests1-linux64-build63.txt.gz

17:05:42     INFO -  SUITE-END | took 715s
17:05:42     INFO -  Result summary:
17:05:42     INFO -  cppunittests INFO | Passed: 90
17:05:42     INFO -  cppunittests INFO | Failed: 0

Comment 5

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/build/mozharness/rev/93058ceb3e4e

Comment 6

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/dd689ba706ab

Comment 7

[Geoff Brown [:gbrown]]

Leaving open in case we want to do something for jit and instrumentation tests. (These are not a priority, but to retire the Pandas completely, we need to do something about the suites running on Cedar/Try.)

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/dd689ba706ab

Comment 9

[Geoff Brown [:gbrown]]

From irc today:

<RyanVM|sheriffduty> kmoir|buildduty: are android 4.3 Jit tests on your radar?
<kmoir|buildduty> RyanVM|sheriffduty: I think gbrown mentioned those and cpp tests were the next ones he was going to green up in the mobile mtg 
<RyanVM|sheriffduty> great
<RyanVM|sheriffduty> we hadn't been running them on 4.0 because they weren't green and nobody took up making them so
<RyanVM|sheriffduty> so would be nice :)

Comment 10

[Geoff Brown [:gbrown]]

cppunit tests run on Android 4.3. Other bugs cover instrumentation and jit-tests. I don't see much purpose to this bug any longer.