1324348 - (stylo-reftest) [meta] stylo: Reftest failures

Status: NEW

(Core :: CSS Parsing and Computation, defect, P5)

Description

[Shing Lyu [:shinglyu]]

I went through the (stylo-vs-gecko) reftest failures and identified the following patterns:

== Things that don't work at all ==
background clip text
bidi
box-properties min/max height
media queries
input/text area/form (the input box is not rendered)
css-invalid/fieldset
css-selector
non-ascii counters
detail-summary (missing the triangle)
generated content
font-inflation
marquee
native-theme
outline
htmlparser (font, line number)
pixel rounding
printing
pagination
svg
tab-size
table-anonymous-boxes (border not rendered)
table border
text-svgglyphs
text-shadow
text-transform
transform

== Things that have some glitches (shift, pixel error) == 
border-dotted
vertical spacing (line height)
mathml (wrong font)
margin-collapsing (vertical scroll bar visible)

== Things that has wrong color ==
Color difference
border-image (color)
box-properties color


You can look at the actual reftest results here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=501669dae26a902f18b8a6154201a9199520fd48&selectedJob=32805142<Paste>

Comment 1

[Shing Lyu [:shinglyu]]

Bobby, do you mind taking a look at these and let me know which are known and which are new so I can file separate bugs for them?

Comment 2

[Shing Lyu [:shinglyu]]

The most common issue is the font is not correct, and the various input tags (<input>, <button>, <textarea>, etc.) does not render at all, fixing those will make a lot of test green.

Comment 3

[Bobby Holley (:bholley)]

I think Cameron is probably more familiar with our existing issues at this point.

Comment 4

[Shing Lyu [:shinglyu]]

Manish, here are some failures for you, there are more coming.

Comment 5

[Shing Lyu [:shinglyu]]

Attachment: crashes.txt

Here are some more crashes!

Comment 6

[Shing Lyu [:shinglyu]]

Attachment: fails_excluding_intermittents.txt

Here are some new failures, excluding the intermittents.

Comment 7

[Shing Lyu [:shinglyu]]

Attachment: intermittent.txt

Here are the intermittents.

The form/input/ tests and the web-animations are particular flaky.

Comment 8

[Shing Lyu [:shinglyu]]

Attachment: timeout.txt

Here are some tests that tend to time-out.

Comment 9

[Boris Zbarsky [:bzbarsky]]

Attachment: Bug 1324348. Reenable various stylo dom reftests.

Review commit: https://reviewboard.mozilla.org/r/114548/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/114548/

Comment 10

[Shing Lyu [:shinglyu]]

Here is a summary of the try run 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f264e04c3b6c85d91975b17c462a4c79bcd4fa92

I'll file bugs for those missing properties that is off our radar first, then try to look into the "Uninvestigated" category.

Uninvestigated

871, Input element styling missing
275, MathML font
245, Writing mode error
172, Flexbox layout is wrong
140, Table alignment
124, Transform not working
114, Not investigated, no reference image
113, Canvas not working
97, Mask not working
93, No scroll bar
88, Gradient not working
88, generated content
86, Line spacing
77, Various SVG bugs
59, Text related layout error
57, Auto-hyphenation bugs
55, detail summary not implemented
48, Border bugs
47, test images are blank
45, Color difference
37, Image element
31, Un-investigated layout error
24, Scoped style?
24, Animation not implemented
22, Pagination?
20, Values error
19, Box property bugs
17, Ruby layout wrong
15, ua-stylesheet
14, image-rect
14, Focus is not working
14, Background and border image issue
13, Web component not working
13, Text orientation is wrong
13, Text alignment is wrong
11, Webkit-gradient
11, Multiple insert/append?
11, Border image repeat is wrong
10, Web animation
10, Printing bugs(missing page etc)
9, Text color is wrong
9, Source code view missing line number
9, small cap
9, css transition not implemented
8, Display bugs
8, Box shadow bugs
8, Background color missing
7, Need marquee
7, Missing space between background patterns
6, CSS Selector not implemented
6, Bad background pattern size
5, webkit-box
5, Text justification is wrong
5, Ruby not working
5, Pixel rounding error
5, multicol not working
4, Plugin bugs
4, Media query not implemented
3, Wrong clipping
3, Import
3, Blending bugs
2, Float issues
0, 

Known Unimplemented Properties

580, Wrong font or wrong font size
297, Wrong font (Ahem)
266, grid not implemented
175, css counter not implemented
131, Image size or positioning wrong (image-orientation)
79, Shape outside not working (shape-outside)
42, Image blurred (zoom?)
18, will-change errors (will-change)
6, List style wrong (list-style-type)
5, Page breaking is wrong (break-before|after?)
motion?
initial-letter, initial-letter-align

Comment 11

[Shing Lyu [:shinglyu]]

Attachment: Reftest failures categorized

Test case list for the previous comment.

Comment 12

[Shing Lyu [:shinglyu]]

Latest try, rebased to Mar 27's tip https://treeherder.mozilla.org/#/jobs?repo=try&revision=de5b2653facb3e314126ac1c8f768391a9a03fa6

Comment 13

[Shing Lyu [:shinglyu]]

My primary goal is to identify key issues that causes a lot of failures, so I might miss a few tests because I only skim through them. I also gave up on the bugs folder, they are almost all fail for different reasons.

Comment 14

[Astley Chen (inactive)]

elevate to P3 as it's time to clean them up one by one.

Comment 15

[Bobby Holley (:bholley)]

Thank you for doing this analysis shing - it is incredibly helpful for prioritization!