Closed Bug 865006 Opened 8 years ago Closed 6 years ago

Run more xpcshell tests on Android

Categories

(Firefox for Android :: General, defect)

x86
Android
defect
Not set
normal

Tracking

()

RESOLVED FIXED
Firefox 34

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

Attachments

(6 files)

Android xpcshell is running reliably now, but only runs one test. We can run many more with just a little effort.
These all pass locally. Just need to get the modules dir set correctly in automated runs before checking this in.
Depends on: 861295
Comment on attachment 741035 [details] [diff] [review]
First patch: adds chrome, locale, netwerk, zipwriter, xml, image, xpcshell example, xpcom, rdf

This patch updates xpcshell manifests to run more android tests.

Runs fine on try:

https://tbpl.mozilla.org/?tree=Try&rev=9ff36c69d1c4

There is nothing special about this particular subset of tests: It's just the ones I started with. I will try enabling more tests in batches in the coming weeks.

I intend to update the bugs associated with modified skip-if/fail-if's (eg, bug 675039) once this lands.
Attachment #741035 - Flags: review?(jmaher)
Comment on attachment 741035 [details] [diff] [review]
First patch: adds chrome, locale, netwerk, zipwriter, xml, image, xpcshell example, xpcom, rdf

Review of attachment 741035 [details] [diff] [review]:
-----------------------------------------------------------------

I would like to see the try server run for this and see 10 greens in a row if possible.
Attachment #741035 - Flags: review?(jmaher) → review+
First patch: https://hg.mozilla.org/integration/mozilla-inbound/rev/a0e3982ba3fb

Typical output:

INFO | Passed: 261
INFO | Failed: 0
INFO | Todo: 15
program finished with exit code 0
elapsedTime=913.717827
Whiteboard: [leave open]
Attachment #741035 - Attachment description: wip → First patch: adds chrome, locale, netwerk, zipwriter, xml, image, xpcshell example, xpcom, rdf
That started to fail on inbound, for android armv6 in the netwerk tests -- I disabled all of the netwerk tests again: 

https://hg.mozilla.org/integration/mozilla-inbound/rev/0c6b9bedfa65
Comment on attachment 745730 [details] [diff] [review]
(2) Add more xpcshell tests - mostly dom

Review of attachment 745730 [details] [diff] [review]:
-----------------------------------------------------------------

wow, we are about half way there.
Attachment #745730 - Flags: review?(jmaher) → review+
Comment on attachment 747951 [details] [diff] [review]
(3) Add more xpcshell tests

Review of attachment 747951 [details] [diff] [review]:
-----------------------------------------------------------------

wow, are we close to the end here?
Attachment #747951 - Flags: review?(jmaher) → review+
We are making progress, but there's still another 60 or so manifests listed in xpcshell.ini not yet in xpcshell_android.ini.


(3) https://hg.mozilla.org/integration/mozilla-inbound/rev/1c84eeff5a68
This should not have landed with DONTBUILD...
Quick status update: I am holding off on enabling more tests until xpcshell tests are running on Android 4.0/panda -- bug 874965. Progress on that bug is waiting on the ability to run Android 4.0 X on try -- bug 881293. AFAIK, none of this is a high priority; if that's not the case, please comment.
Depends on: 874965
xpcshell tests for Panda/Android 4.0 can be run, hidden, on try now. 

This patch adds a few more manifests and skips various tests for failures on the various Android platforms. The result runs clean on Android 2.2, Android armv6, Android 4.0, and Android x86 (coming soon!). 

https://tbpl.mozilla.org/?tree=Try&rev=8acbac0913ad&showall=1

I hope to wrap up this bug with 2 or 3 more patches.
Attachment #794979 - Flags: review?(jmaher)
Comment on attachment 794979 [details] [diff] [review]
(4) Add more xpcshell tests; skip a few for panda and x86

Review of attachment 794979 [details] [diff] [review]:
-----------------------------------------------------------------

this looks like a nice addition to our xpcshell tests!
Attachment #794979 - Flags: review?(jmaher) → review+
Depends on: 911836
I resurrected the wip patches, updated as necessary, and verified on try:

https://tbpl.mozilla.org/?tree=Try&rev=5bef6486d53f

This feels like a good place to close this bug. About half of all xpcshell tests now run on Android. Many of the remaining manifests require substantial skip-if/fail-if annotations for Android, and often tests fail on Android simply because the feature under test is not implemented, or is implemented differently, on Android. We've established a good framework here and shown that xpcshell tests can be run; I'll leave it to developers to add more tests as desired. 

https://hg.mozilla.org/integration/mozilla-inbound/rev/ddda7acf3739
Whiteboard: [leave open]
https://hg.mozilla.org/mozilla-central/rev/ddda7acf3739
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 34
You need to log in before you can comment on or make changes to this bug.