1473368 - Review fennec emulator unit tests

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Testing, enhancement, P2)

Description

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Fennec unit tests - like robocop, mochitests, reftests - run in continuous integration on emulators on aws and have run with little modification over the last year.

Platforms are seen on treeherder as:
Android 4.2 x86 opt
Android 4.3 API16+ opt
Android 4.3 API16+ debug

A full test set amounts to about 300 tasks, although that includes 140 js-reftests, which are often optimized out.

With the advent of bitbar and packet.net becoming available this year, we should evaluate:
 - can some of the existing tests be eliminated?
 - can some of the existing tests be moved to packet.net, or perhaps bitbar?

Comment 1

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 2

[Geoff Brown [:gbrown] (pto Apr 11-18)]

Victories:

• retired 4.2 x86 test platform
• retired geckoview_example smoketest
• moved Marionette tests from Android 4.3 to Android x86_64 7.0

Pending victories:

• cppunit, xpcshell, and geckoview-junit can be moved from Android 4.3 to x86_64 7.0 once blocking issues are resolved

We have also had great discussions about the remaining Android 4.3 tests.

• robocop, mochitests, reftests, and crashtests provide important test coverage for fennec; they cannot be discontinued until fennec support is discontinued
• jsreftests provide important test coverage for core js code on armv7; going forward, we will arrange for capacity on bitbar to run these tests on armv7 hardware, likely via geckoview TRA; until then, these tests will continue to run on the 4.3 emulator.

It is anticipated that fennec support will be discontinued (except for esr) later this year, at which time any remaining 4.3 emulator tests will be stopped on trunk.

Comment 3

[Geoff Brown [:gbrown] (pto Apr 11-18)]

There's no "active" work remaining here -- we just need to wait for blocking issues to resolve themselves, as covered by other bugs.