I think this is an important omission. It's enabled on b2g.
Here is a try run with about 60 failures https://tbpl.mozilla.org/?tree=Try&rev=7f601f035eac A good chunk are unexpected passes, I think most of the failures are annotated as skip or fails on b2g already, and there are some after that to look at still.
this might be a quick fix. Our bar for turning on new android tests is to run each test suite (chunk in this case) 10 times and have consistent results.
first pass at fixing the manifest: https://tbpl.mozilla.org/?tree=Try&rev=3d878ead1491
Created attachment 746960 [details] [diff] [review] fix the bugs/reftest.list manifest to support android (1.0) this is my second attempt to fix this. My first attempt missed a couple instances. This patch is up on try server right now: https://tbpl.mozilla.org/?tree=Try&rev=76298dc34770 If this looks good we can review and turn on hundreds of additional reftests for android.
Created attachment 747199 [details] [diff] [review] fix the bugs/reftest.list manifest to support android (2.0) Looking good on try server: https://tbpl.mozilla.org/?tree=Try&rev=4ea6ddeeeaa1
Comment on attachment 747199 [details] [diff] [review] fix the bugs/reftest.list manifest to support android (2.0) Yay! Thanks Joel for picking this up so quickly that I didn't even have a chance to get to it. It's great that we'll have coverage for one of the most important reftest directories on android. Any particular reason that the first few annotations are skip-if(Android) and the rest are fails-if(Android)? I think fails-if is better, but that's a nit. I'm going to try to go through the tests that fail and see if any of them point out any major problems that we should have bugs filed on.
Attachment #747199 - Flags: review?(tnikkel) → review+
Created attachment 747432 [details] [diff] [review] fix the bugs/reftest.list manifest to support android (2.1) Updated to change most of the skip-if to fails-if and tested on try! In all my testing I have seen a couple random failures crop up from this, we are looking at those being really random orange, not some 20%+ of the time thing.
I marked two fails-if as random-if because they were passing sometimes. https://hg.mozilla.org/integration/mozilla-inbound/rev/c3626167f6d8
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla23
753329-1.html also likes to pass from time to time. https://hg.mozilla.org/integration/mozilla-inbound/rev/d72307b5972b
You need to log in before you can comment on or make changes to this bug.