1585119 - Review mochitest tests skipped on Android

Status: RESOLVED FIXED

(Testing :: General, task, P2)

Description

[Geoff Brown [:gbrown]]

Some mochitest tests currently skipped on Android were skipped because they failed in the android test environment of the time: the android 4.3 emulator, or perhaps a pandaboard or tegraboard. Today's emulator is much faster and has other improved capabilities. Also, our code has evolved. This seems like a good time to review skipped tests: Can some of them be enabled without introducing new failures?

Comment 1

[Geoff Brown [:gbrown]]

Attachment: Bug 1585119 - Re-enable many mochitests on android; r=

Most of these tests have been disabled for a long time; they run well
in the current test environment.
With the additional tests running, task times increase; I have added one
more test chunk for android mochitest-plain.
These tests were identified from a random sampling of mochitest manifests;
I intend to enable more mochitests in future patches.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/10a01cfef712
Re-enable many mochitests on android; r=geckoview-reviewers,snorp

Comment 3

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/10a01cfef712

Comment 4

[Geoff Brown [:gbrown]]

Before:
./mach test-info report --filter-keys=skip-if --filter-values=android --show-summary --flavor=mochitest
Finding tests...
Resolver found 10360 tests, 160 manifests
{
  "summary": {
    "components": 54, 
    "failed tests": 243, 
    "manifests": 160, 
    "skipped tests": 4975, 
    "tests": 4975
  }
}
After:
Resolver found 10350 tests, 157 manifests
{
  "summary": {
    "components": 51, 
    "failed tests": 238, 
    "manifests": 157, 
    "skipped tests": 4850, 
    "tests": 4850
  }
}

Comment 5

[Edwin Takahashi (:egao | infrequent contributor)]

Nice reduction in test cases, and I like that command for my future work as well.

Comment 6

[Geoff Brown [:gbrown]]

Attachment: Bug 1585119 - Re-enable many more mochitests on Android; r=

Most of these tests have been disabled for a long time; they run well
in the current test environment.
I intend to enable still more mochitests in a future patch.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/da4ef1bcce4c
Re-enable many more mochitests on Android; r=geckoview-reviewers,snorp

Comment 8

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/da4ef1bcce4c

Comment 9

[Geoff Brown [:gbrown]]

Final planned patch ready now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6150ed322df5c324fb786b2950cba18fc2495834

Resolver found 10351 tests, 157 manifests
{
"summary": {
"components": 44,
"failed tests": 235,
"manifests": 157,
"skipped tests": 4592,
"tests": 4592
}
}

4975 - 4592 = 383 mochitests un-skipped.

Comment 10

[Geoff Brown [:gbrown]]

Attachment: Bug 1585119 - Re-enable yet more mochitests on android; r=

Most of these tests have been disabled for a long time; they run well
in the current test environment.
This completes my review of skipped Android tests.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8b0d130f24d6
Re-enable yet more mochitests on android; r=geckoview-reviewers,snorp

Comment 12

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/8b0d130f24d6