enable layout/reftests/bugs on android

RESOLVED FIXED in mozilla23

Status

()

Core
Layout
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: tnikkel, Assigned: jmaher)

Tracking

Trunk
mozilla23
x86_64
Linux
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

5 years ago
I think this is an important omission.

It's enabled on b2g.
(Reporter)

Comment 1

5 years ago
Here is a try run with about 60 failures
https://tbpl.mozilla.org/?tree=Try&rev=7f601f035eac
A good chunk are unexpected passes, I think most of the failures are annotated as skip or fails on b2g already, and there are some after that to look at still.
(Assignee)

Comment 2

5 years ago
this might be a quick fix.  Our bar for turning on new android tests is to run each test suite (chunk in this case) 10 times and have consistent results.
(Assignee)

Comment 3

5 years ago
first pass at fixing the manifest:
https://tbpl.mozilla.org/?tree=Try&rev=3d878ead1491
(Assignee)

Comment 4

5 years ago
Created attachment 746960 [details] [diff] [review]
fix the bugs/reftest.list manifest to support android (1.0)

this is my second attempt to fix this.  My first attempt missed a couple instances.  This patch is up on try server right now:
https://tbpl.mozilla.org/?tree=Try&rev=76298dc34770

If this looks good we can review and turn on hundreds of additional reftests for android.
(Assignee)

Comment 5

5 years ago
Created attachment 747199 [details] [diff] [review]
fix the bugs/reftest.list manifest to support android (2.0)

Looking good on try server:
https://tbpl.mozilla.org/?tree=Try&rev=4ea6ddeeeaa1
Assignee: tnikkel → jmaher
Attachment #746960 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #747199 - Flags: review?(tnikkel)
(Reporter)

Comment 6

5 years ago
Comment on attachment 747199 [details] [diff] [review]
fix the bugs/reftest.list manifest to support android (2.0)

Yay! Thanks Joel for picking this up so quickly that I didn't even have a chance to get to it.

It's great that we'll have coverage for one of the most important reftest directories on android.

Any particular reason that the first few annotations are skip-if(Android) and the rest are fails-if(Android)? I think fails-if is better, but that's a nit.

I'm going to try to go through the tests that fail and see if any of them point out any major problems that we should have bugs filed on.
Attachment #747199 - Flags: review?(tnikkel) → review+
(Assignee)

Comment 7

5 years ago
Created attachment 747432 [details] [diff] [review]
fix the bugs/reftest.list manifest to support android (2.1)

Updated to change most of the skip-if to fails-if and tested on try!  In all my testing I have seen a couple random failures crop up from this, we are looking at those being really random orange, not some 20%+ of the time thing.
Attachment #747199 - Attachment is obsolete: true
Attachment #747432 - Flags: review+
(Assignee)

Updated

5 years ago
Keywords: checkin-needed
Depends on: 870620
(Reporter)

Comment 9

5 years ago
I marked two fails-if as random-if because they were passing sometimes.
https://hg.mozilla.org/integration/mozilla-inbound/rev/c3626167f6d8
https://hg.mozilla.org/mozilla-central/rev/75d1409ae090
https://hg.mozilla.org/mozilla-central/rev/c3626167f6d8
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla23
753329-1.html also likes to pass from time to time.
https://hg.mozilla.org/integration/mozilla-inbound/rev/d72307b5972b

Updated

5 years ago
Depends on: 870764
You need to log in before you can comment on or make changes to this bug.