Bug 1324348 (stylo-reftest)

[meta] stylo: Reftest failures

NEW
Unassigned

Status

()

defect
P5
normal
3 years ago
6 months ago

People

(Reporter: shinglyu, Unassigned)

Tracking

(Depends on 2 bugs, Blocks 1 bug, {meta})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 2 obsolete attachments)

Reporter

Description

3 years ago
I went through the (stylo-vs-gecko) reftest failures and identified the following patterns:

== Things that don't work at all ==
background clip text
bidi
box-properties min/max height
media queries
input/text area/form (the input box is not rendered)
css-invalid/fieldset
css-selector
non-ascii counters
detail-summary (missing the triangle)
generated content
font-inflation
marquee
native-theme
outline
htmlparser (font, line number)
pixel rounding
printing
pagination
svg
tab-size
table-anonymous-boxes (border not rendered)
table border
text-svgglyphs
text-shadow
text-transform
transform

== Things that have some glitches (shift, pixel error) == 
border-dotted
vertical spacing (line height)
mathml (wrong font)
margin-collapsing (vertical scroll bar visible)

== Things that has wrong color ==
Color difference
border-image (color)
box-properties color


You can look at the actual reftest results here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=501669dae26a902f18b8a6154201a9199520fd48&selectedJob=32805142<Paste>
Reporter

Updated

3 years ago
Blocks: stylo
Reporter

Comment 1

3 years ago
Bobby, do you mind taking a look at these and let me know which are known and which are new so I can file separate bugs for them?
Flags: needinfo?(bobbyholley)
Reporter

Comment 2

3 years ago
The most common issue is the font is not correct, and the various input tags (<input>, <button>, <textarea>, etc.) does not render at all, fixing those will make a lot of test green.
I think Cameron is probably more familiar with our existing issues at this point.
Flags: needinfo?(bobbyholley) → needinfo?(cam)
Reporter

Comment 4

2 years ago
Manish, here are some failures for you, there are more coming.
Flags: needinfo?(manishearth)
Flags: needinfo?(manishearth)
Comment hidden (offtopic)
Reporter

Comment 6

2 years ago
Here are some new failures, excluding the intermittents.
Attachment #8823987 - Attachment is obsolete: true
Reporter

Comment 7

2 years ago
Posted file intermittent.txt
Here are the intermittents.

The form/input/ tests and the web-animations are particular flaky.
Reporter

Comment 8

2 years ago
Posted file timeout.txt
Here are some tests that tend to time-out.
Blocks: stylo-nightly
No longer blocks: stylo
Priority: -- → P1
Depends on: 1323643
Flags: needinfo?(cam)
Alias: stylo-reftest
Comment hidden (mozreview-request)
Attachment #8839970 - Attachment is obsolete: true
Attachment #8839970 - Flags: review?(bobbyholley)
Priority: P1 → P5
Depends on: 1342591
Depends on: 1337068
Depends on: 1321769
Depends on: 1347065
Reporter

Comment 10

2 years ago
Here is a summary of the try run 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f264e04c3b6c85d91975b17c462a4c79bcd4fa92

I'll file bugs for those missing properties that is off our radar first, then try to look into the "Uninvestigated" category.

Uninvestigated

871, Input element styling missing
275, MathML font
245, Writing mode error
172, Flexbox layout is wrong
140, Table alignment
124, Transform not working
114, Not investigated, no reference image
113, Canvas not working
97, Mask not working
93, No scroll bar
88, Gradient not working
88, generated content
86, Line spacing
77, Various SVG bugs
59, Text related layout error
57, Auto-hyphenation bugs
55, detail summary not implemented
48, Border bugs
47, test images are blank
45, Color difference
37, Image element
31, Un-investigated layout error
24, Scoped style?
24, Animation not implemented
22, Pagination?
20, Values error
19, Box property bugs
17, Ruby layout wrong
15, ua-stylesheet
14, image-rect
14, Focus is not working
14, Background and border image issue
13, Web component not working
13, Text orientation is wrong
13, Text alignment is wrong
11, Webkit-gradient
11, Multiple insert/append?
11, Border image repeat is wrong
10, Web animation
10, Printing bugs(missing page etc)
9, Text color is wrong
9, Source code view missing line number
9, small cap
9, css transition not implemented
8, Display bugs
8, Box shadow bugs
8, Background color missing
7, Need marquee
7, Missing space between background patterns
6, CSS Selector not implemented
6, Bad background pattern size
5, webkit-box
5, Text justification is wrong
5, Ruby not working
5, Pixel rounding error
5, multicol not working
4, Plugin bugs
4, Media query not implemented
3, Wrong clipping
3, Import
3, Blending bugs
2, Float issues
0, 

Known Unimplemented Properties

580, Wrong font or wrong font size
297, Wrong font (Ahem)
266, grid not implemented
175, css counter not implemented
131, Image size or positioning wrong (image-orientation)
79, Shape outside not working (shape-outside)
42, Image blurred (zoom?)
18, will-change errors (will-change)
6, List style wrong (list-style-type)
5, Page breaking is wrong (break-before|after?)
motion?
initial-letter, initial-letter-align
Reporter

Comment 11

2 years ago
Test case list for the previous comment.
Assignee: nobody → slyu
Reporter

Updated

2 years ago
Assignee: slyu → nobody
Reporter

Updated

2 years ago
Summary: [Stylo] Reftest failures → stylo: Reftest failures
Reporter

Comment 13

2 years ago
My primary goal is to identify key issues that causes a lot of failures, so I might miss a few tests because I only skim through them. I also gave up on the bugs folder, they are almost all fail for different reasons.
Reporter

Updated

2 years ago
Depends on: 1351548
Depends on: 1351724
No longer depends on: 1351548
Reporter

Updated

2 years ago
Depends on: 1351971
Reporter

Updated

2 years ago
Depends on: 1351973
Reporter

Updated

2 years ago
Depends on: 1351976
Reporter

Updated

2 years ago
Depends on: 1351978
elevate to P3 as it's time to clean them up one by one.
Priority: P5 → P3
Reporter

Updated

2 years ago
Depends on: stylo-svg
Reporter

Updated

2 years ago
Depends on: 1352285
Reporter

Updated

2 years ago
Depends on: 1352291
Reporter

Updated

2 years ago
Depends on: 1340484
Thank you for doing this analysis shing - it is incredibly helpful for prioritization!
Reporter

Updated

2 years ago
Depends on: 1353994
Reporter

Updated

2 years ago
Depends on: 1354000
Reporter

Updated

2 years ago
Depends on: 1354404
Reporter

Updated

2 years ago
No longer depends on: 1354404
Reporter

Updated

2 years ago
Depends on: 1354406
No longer depends on: 1354406
Reporter

Updated

2 years ago
Depends on: 1354999
Reporter

Updated

2 years ago
Depends on: 1355003
Depends on: 1358812
Depends on: 1360424
Depends on: 1360784
Depends on: 1361045
Reporter

Updated

2 years ago
Depends on: 1361645
Reporter

Updated

2 years ago
Depends on: 1365510
Reporter

Updated

2 years ago
No longer depends on: 1365510
Reporter

Updated

2 years ago
Depends on: 1365822
Reporter

Updated

2 years ago
No longer depends on: 1365822
Reporter

Updated

2 years ago
Depends on: 1366144
Reporter

Updated

2 years ago
No longer depends on: 1366144
No longer depends on: 1337655
Reporter

Updated

2 years ago
Depends on: 1366977
Reporter

Updated

2 years ago
No longer depends on: 1366977
Reporter

Updated

2 years ago
Depends on: 1367345
Reporter

Updated

2 years ago
Depends on: 1367636
Depends on: 1367923
Depends on: 1367947
Depends on: 1370358
Depends on: 1371030
Depends on: 1371034
Depends on: 1371036
Reporter

Updated

2 years ago
Depends on: 1371194
No longer blocks: stylo
Depends on: 1377469
Depends on: 1380934
Depends on: 1383869
Priority: P3 → --
Keywords: meta
Priority: -- → P5
Summary: stylo: Reftest failures → [meta] stylo: Reftest failures
Depends on: 1386733
Depends on: 1388911
Depends on: 1393645
Depends on: 1396135
Depends on: 1396136
Depends on: 1396147
Depends on: 1396151
Depends on: 1396411
Depends on: 1396977
You need to log in before you can comment on or make changes to this bug.