973835 - (B2GRT) [META] Enable all skip reftest cases on B2G

Status: RESOLVED WONTFIX

(Core :: Graphics, defect)

Description

[u459114]

Create a META bug to collect root causes of reftest fail in each reftest module.

Bug 922680 is the origin bug. In that bug, we intend to enable reftests that can pass after OOP.

Comment 1

[Andrew Halberstadt [:ahal]]

How come a bunch of the dependents were RESOLVED WONTFIX? Are they no longer failing? If not we should keep them open, even if they are being skipped in the manifests to keep better track of what needs to be re-enabled.

Comment 2

[Vincent Chen [:vichen]]

(In reply to Andrew Halberstadt [:ahal] from comment #1)
> How come a bunch of the dependents were RESOLVED WONTFIX? Are they no longer
> failing? If not we should keep them open, even if they are being skipped in
> the manifests to keep better track of what needs to be re-enabled.

As this week is my last week at Mozilla, so I mark RESOVLED WONTFIX to the bugs which may confuse others. The issues I mark RESOLVED WONTFIX are the fail by folder and the log is out of date and it tells us few. We keep single fail by reftest, like Bug 988758. Roughly estimate, there are about 2000 reftests not pass (fails-if, skip-if and random-if) now and I suggest to CJ to review the random-if first. The random-if should not problem of the render engine itself, but affect by others (like the fails due to scrolling we met before).

Comment 3

[u459114]

Clone bug 1000722 comment 42 

Try result of rolling back random-if patches in bug 981477
https://tbpl.mozilla.org/?tree=Try&rev=b360e1cc611c 

1. A skip item which is also fixed by this patch
a. skip-if(B2G&&browserIsRemote) == box-ordinal-with-out-of-flow-1.html ...

2. Need to file a bug to trace status
layout/reftests/bugs/reftest.list
[Unknown] random-if(Android||(B2G&&browserIsRemote)) == 728983-1.html 728983-1-ref.html 
[scrollbar] random-if(B2G&&browserIsRemote) == 960822.html 960822-ref.html..
[scrollbar] random-if(B2G&&browserIsRemote) == 381497-n.html 381497-f.html ..

3. There are associated bugs on XP. But we may still need to file issues for B2G
[Bug 943863 XP] random-if(B2G&&browserIsRemote) == box-sizing-replaced-002.xht ..
[Bug 935850 XP] random-if(B2G&&browserIsRemote) == box-sizing-replaced-003.xht ..
[Bug 629416 XP] random-if(B2G&&browserIsRemote) == 413928-1.html 413928-1-ref.html
[Bug 629416 XP] random-if(B2G&&browserIsRemote) == 413928-2.html 413928-2-ref.html

4. Know issues which still not assign
[Bug 982547] random-if(Android) skip-if(B2G&&browserIsRemote) == box-sizing-replaced-001.xht..
[Bug 988759] random-if(B2G&&browserIsRemote) == multipleinsertionpoints-ref2.xhtml ...
[Bug 988763] random-if(B2G&&browserIsRemote) == referenced-from-binding-01.html ...
[Bug 988765] random-if(B2G&&browserIsRemote) == 166591-dynamic-1.html ...
[Bug 972697] skip-if(B2G&&browserIsRemote) include font-inflation/reftest.list

Comment 4

[u459114]

> 2. Need to file a bug to trace status
> layout/reftests/bugs/reftest.list
> [Unknown] random-if(Android||(B2G&&browserIsRemote)) == 728983-1.html
> 728983-1-ref.html 
> [scrollbar] random-if(B2G&&browserIsRemote) == 960822.html 960822-ref.html..
> [scrollbar] random-if(B2G&&browserIsRemote) == 381497-n.html 381497-f.html ..
Bug filed: 1031672 1031673 1031674

Comment 5

[u459114]

(In reply to C.J. Ku[:cjku] from comment #3)
> 3. There are associated bugs on XP. But we may still need to file issues for
> B2G
> [Bug 943863 XP] random-if(B2G&&browserIsRemote) ==
> box-sizing-replaced-002.xht ..
> [Bug 935850 XP] random-if(B2G&&browserIsRemote) ==
> box-sizing-replaced-003.xht ..
> [Bug 629416 XP] random-if(B2G&&browserIsRemote) == 413928-1.html
> 413928-1-ref.html
> [Bug 629416 XP] random-if(B2G&&browserIsRemote) == 413928-2.html
> 413928-2-ref.html
Bug filed: 1031676, 1031677

Comment 6

[u459114]

Types of failures
1. XBL test cases - solution: find it out and skip-if
2. font-inflation - I am figuring out how to make reftest plug-in not an "app". In current design(retest.js/ reftest-content.js), it's indeed an app. Since it's an app, font-inflation is been disabled, we can't test it on B2G at all
3. timing issue: I am looking into DrawWindow flow. One thing that I need to understand is when we call nsDocument::FlushPendingNotifications at chrome process, will it clean up all pending paint task in content process. If the answer is no, then, that might be the root cause of it.
4. Misc

Comment 7

[u459114]

bug 1009628 might fix #3

Comment 8

[u459114]

Unknown
REFTEST TEST-UNEXPECTED-FAIL | http://10.247.24.74:8888/tests/layout/reftests/svg/moz-only/foreignObject-zoom-01.svg | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | http://10.247.24.74:8888/tests/layout/reftests/svg/moz-only/zoom-invalidation-01.svg | image comparison (==)

CJK rendering issue, unknow
REFTEST TEST-UNEXPECTED-FAIL | http://10.247.24.74:8888/tests/layout/reftests/svg/text-language-00.xhtml | image comparison (!=)
REFTEST TEST-UNEXPECTED-FAIL | http://10.247.24.74:8888/tests/layout/reftests/svg/text-language-01.xhtml | image comparison (!=)

Two unknown issues

Comment 9

[u459114]

Reason -- unknown
 17 # Tests for zooming with the full page zoom UI
 18 skip-if(B2G) random-if(d2d) == feImage-zoom-01a.svg          feImage-zoom-01-ref.svg
 19 skip-if(B2G) random-if(d2d) == feImage-zoom-01b.svg          feImage-zoom-01-ref.svg
 20 skip-if(B2G) == foreignObject-zoom-01.svg                    pass.svg
 21 skip-if(B2G) == zoom-invalidation-01.svg                     pass.svg

Comment 10

[u459114]

Timing issue
TEST-UNEXPECTED-FAIL | http://10.247.24.74:8888/tests/layout/reftests/svg/moz-only/zoom-invalidation-01.svg

Comment 11

[u459114]

Since we already enable B2G OOP reftest for a while, change this META bug to trace all disable B2G reftest

Eventually, all test cases should be enable on B2G, unless that one is not valid on B2G, such as XBL/XUL test case, touch event with combined key.

Comment 12

[u459114]

Depend on bug 1048624 first, and figure out which of them are reftest

Comment 13

[u459114]

Looks like there are many unfix issues which are depend on bug 773482.

Comment 14

[u459114]

Attachment: reftest-healthy

Comment 15

[u459114]

Attachment: reftest-healthy

Comment 16

[u459114]

Attachment: reftest-healthy

Comment 17

[u459114]

Attachment: reftest-healthy

Comment 18

[u459114]

* Failure type : skip-if.
* Platform     : B2G.
* Total match  : 427.

* Failure type : random-if.
* Platform     : B2G.
* Total match  : 19.

* Failure type : skip-if.
* Platform     : Android.
* Total match  : 71.

* Failure type : random-if.
* Platform     : Android.
* Total match  : 89.

Comment 19

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This is WONTFIX (i.e. we're not going to invest in enabling/fixing reftests on B2G), now that we're removing B2G code from the tree via bug 1306391.