Bug 1324348 (stylo-reftest)

[meta] stylo: Reftest failures

NEW
Unassigned

Status

()

Core
CSS Parsing and Computation
P5
normal
2 years ago
11 months ago

People

(Reporter: shinglyu, Unassigned)

Tracking

(Depends on: 4 bugs, Blocks: 1 bug, {meta})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments, 2 obsolete attachments)

(Reporter)

Description

2 years ago
I went through the (stylo-vs-gecko) reftest failures and identified the following patterns:

== Things that don't work at all ==
background clip text
bidi
box-properties min/max height
media queries
input/text area/form (the input box is not rendered)
css-invalid/fieldset
css-selector
non-ascii counters
detail-summary (missing the triangle)
generated content
font-inflation
marquee
native-theme
outline
htmlparser (font, line number)
pixel rounding
printing
pagination
svg
tab-size
table-anonymous-boxes (border not rendered)
table border
text-svgglyphs
text-shadow
text-transform
transform

== Things that have some glitches (shift, pixel error) == 
border-dotted
vertical spacing (line height)
mathml (wrong font)
margin-collapsing (vertical scroll bar visible)

== Things that has wrong color ==
Color difference
border-image (color)
box-properties color


You can look at the actual reftest results here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=501669dae26a902f18b8a6154201a9199520fd48&selectedJob=32805142<Paste>
(Reporter)

Updated

2 years ago
Blocks: 1243581
(Reporter)

Comment 1

2 years ago
Bobby, do you mind taking a look at these and let me know which are known and which are new so I can file separate bugs for them?
Flags: needinfo?(bobbyholley)
(Reporter)

Comment 2

2 years ago
The most common issue is the font is not correct, and the various input tags (<input>, <button>, <textarea>, etc.) does not render at all, fixing those will make a lot of test green.
I think Cameron is probably more familiar with our existing issues at this point.
Flags: needinfo?(bobbyholley) → needinfo?(cam)
(Reporter)

Comment 4

2 years ago
Manish, here are some failures for you, there are more coming.
Flags: needinfo?(manishearth)
Flags: needinfo?(manishearth)
Comment hidden (offtopic)
(Reporter)

Comment 6

2 years ago
Created attachment 8823989 [details]
fails_excluding_intermittents.txt

Here are some new failures, excluding the intermittents.
Attachment #8823987 - Attachment is obsolete: true
(Reporter)

Comment 7

2 years ago
Created attachment 8823991 [details]
intermittent.txt

Here are the intermittents.

The form/input/ tests and the web-animations are particular flaky.
(Reporter)

Comment 8

2 years ago
Created attachment 8823992 [details]
timeout.txt

Here are some tests that tend to time-out.
Blocks: 1330412
No longer blocks: 1243581
Priority: -- → P1
Depends on: 1323643
Depends on: 1334579, 1303747
Flags: needinfo?(cam)
Comment hidden (mozreview-request)
Attachment #8839970 - Attachment is obsolete: true
Attachment #8839970 - Flags: review?(bobbyholley)
Priority: P1 → P5
Depends on: 1337068
Depends on: 1321769
Depends on: 1347065
(Reporter)

Comment 10

a year ago
Here is a summary of the try run 

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f264e04c3b6c85d91975b17c462a4c79bcd4fa92

I'll file bugs for those missing properties that is off our radar first, then try to look into the "Uninvestigated" category.

Uninvestigated

871, Input element styling missing
275, MathML font
245, Writing mode error
172, Flexbox layout is wrong
140, Table alignment
124, Transform not working
114, Not investigated, no reference image
113, Canvas not working
97, Mask not working
93, No scroll bar
88, Gradient not working
88, generated content
86, Line spacing
77, Various SVG bugs
59, Text related layout error
57, Auto-hyphenation bugs
55, detail summary not implemented
48, Border bugs
47, test images are blank
45, Color difference
37, Image element
31, Un-investigated layout error
24, Scoped style?
24, Animation not implemented
22, Pagination?
20, Values error
19, Box property bugs
17, Ruby layout wrong
15, ua-stylesheet
14, image-rect
14, Focus is not working
14, Background and border image issue
13, Web component not working
13, Text orientation is wrong
13, Text alignment is wrong
11, Webkit-gradient
11, Multiple insert/append?
11, Border image repeat is wrong
10, Web animation
10, Printing bugs(missing page etc)
9, Text color is wrong
9, Source code view missing line number
9, small cap
9, css transition not implemented
8, Display bugs
8, Box shadow bugs
8, Background color missing
7, Need marquee
7, Missing space between background patterns
6, CSS Selector not implemented
6, Bad background pattern size
5, webkit-box
5, Text justification is wrong
5, Ruby not working
5, Pixel rounding error
5, multicol not working
4, Plugin bugs
4, Media query not implemented
3, Wrong clipping
3, Import
3, Blending bugs
2, Float issues
0, 

Known Unimplemented Properties

580, Wrong font or wrong font size
297, Wrong font (Ahem)
266, grid not implemented
175, css counter not implemented
131, Image size or positioning wrong (image-orientation)
79, Shape outside not working (shape-outside)
42, Image blurred (zoom?)
18, will-change errors (will-change)
6, List style wrong (list-style-type)
5, Page breaking is wrong (break-before|after?)
motion?
initial-letter, initial-letter-align
(Reporter)

Comment 11

a year ago
Created attachment 8851485 [details]
Reftest failures categorized

Test case list for the previous comment.
Assignee: nobody → slyu
(Reporter)

Updated

a year ago
Assignee: slyu → nobody
(Reporter)

Updated

a year ago
Summary: [Stylo] Reftest failures → stylo: Reftest failures
(Reporter)

Comment 13

a year ago
My primary goal is to identify key issues that causes a lot of failures, so I might miss a few tests because I only skim through them. I also gave up on the bugs folder, they are almost all fail for different reasons.
(Reporter)

Updated

a year ago
Depends on: 1351548
Depends on: 1351724
No longer depends on: 1351548
(Reporter)

Updated

a year ago
Depends on: 1351971
(Reporter)

Updated

a year ago
Depends on: 1351973
(Reporter)

Updated

a year ago
Depends on: 1351976
(Reporter)

Updated

a year ago
Depends on: 1351978
elevate to P3 as it's time to clean them up one by one.
Priority: P5 → P3
(Reporter)

Updated

a year ago
Depends on: 1352284
(Reporter)

Updated

a year ago
Depends on: 1352285
(Reporter)

Updated

a year ago
Depends on: 1352291
(Reporter)

Updated

a year ago
Depends on: 1340484
Thank you for doing this analysis shing - it is incredibly helpful for prioritization!
(Reporter)

Updated

a year ago
Depends on: 1353994
(Reporter)

Updated

a year ago
Depends on: 1354000
(Reporter)

Updated

a year ago
Depends on: 1354404
(Reporter)

Updated

a year ago
No longer depends on: 1354404
(Reporter)

Updated

a year ago
Depends on: 1354406
No longer depends on: 1354406
(Reporter)

Updated

a year ago
Depends on: 1354999
(Reporter)

Updated

a year ago
Depends on: 1355003
Depends on: 1358812
Depends on: 1360424
Depends on: 1360784
Depends on: 1361045
(Reporter)

Updated

a year ago
Depends on: 1361645
(Reporter)

Updated

a year ago
Depends on: 1365510
(Reporter)

Updated

a year ago
No longer depends on: 1365510
(Reporter)

Updated

a year ago
Depends on: 1365822
(Reporter)

Updated

a year ago
No longer depends on: 1365822
(Reporter)

Updated

a year ago
Depends on: 1366144
(Reporter)

Updated

a year ago
No longer depends on: 1366144
No longer depends on: 1337655
(Reporter)

Updated

a year ago
Depends on: 1366977
(Reporter)

Updated

a year ago
No longer depends on: 1366977
(Reporter)

Updated

a year ago
Depends on: 1367345
(Reporter)

Updated

a year ago
Depends on: 1367636
Depends on: 1367947
(Reporter)

Updated

a year ago
Depends on: 1371194
No longer blocks: 1243581
Depends on: 1380934
Priority: P3 → --
Keywords: meta
Priority: -- → P5
Summary: stylo: Reftest failures → [meta] stylo: Reftest failures
Depends on: 1388911
Depends on: 1393603
Depends on: 1393645
Depends on: 1394035
Depends on: 1289964
Depends on: 1396044
Depends on: 1396045
Depends on: 1396048
Depends on: 1396135
Depends on: 1396136
Depends on: 1396147
Depends on: 1396151
Depends on: 1396411
Depends on: 1396977
You need to log in before you can comment on or make changes to this bug.