Closed Bug 1585119 Opened 4 months ago Closed 3 months ago

Review mochitest tests skipped on Android

Categories

(Testing :: General, task, P2)

Version 3
task

Tracking

(firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(3 files)

Some mochitest tests currently skipped on Android were skipped because they failed in the android test environment of the time: the android 4.3 emulator, or perhaps a pandaboard or tegraboard. Today's emulator is much faster and has other improved capabilities. Also, our code has evolved. This seems like a good time to review skipped tests: Can some of them be enabled without introducing new failures?

Most of these tests have been disabled for a long time; they run well
in the current test environment.
With the additional tests running, task times increase; I have added one
more test chunk for android mochitest-plain.
These tests were identified from a random sampling of mochitest manifests;
I intend to enable more mochitests in future patches.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/10a01cfef712
Re-enable many mochitests on android; r=geckoview-reviewers,snorp
Before:
./mach test-info report --filter-keys=skip-if --filter-values=android --show-summary --flavor=mochitest
Finding tests...
Resolver found 10360 tests, 160 manifests
{
  "summary": {
    "components": 54, 
    "failed tests": 243, 
    "manifests": 160, 
    "skipped tests": 4975, 
    "tests": 4975
  }
}
After:
Resolver found 10350 tests, 157 manifests
{
  "summary": {
    "components": 51, 
    "failed tests": 238, 
    "manifests": 157, 
    "skipped tests": 4850, 
    "tests": 4850
  }
}

Nice reduction in test cases, and I like that command for my future work as well.

Most of these tests have been disabled for a long time; they run well
in the current test environment.
I intend to enable still more mochitests in a future patch.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/da4ef1bcce4c
Re-enable many more mochitests on Android; r=geckoview-reviewers,snorp

Final planned patch ready now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6150ed322df5c324fb786b2950cba18fc2495834

Resolver found 10351 tests, 157 manifests
{
"summary": {
"components": 44,
"failed tests": 235,
"manifests": 157,
"skipped tests": 4592,
"tests": 4592
}
}

4975 - 4592 = 383 mochitests un-skipped.

Most of these tests have been disabled for a long time; they run well
in the current test environment.
This completes my review of skipped Android tests.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8b0d130f24d6
Re-enable yet more mochitests on android; r=geckoview-reviewers,snorp
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla72
You need to log in before you can comment on or make changes to this bug.