Open Bug 913153 (css3test) Opened 8 years ago Updated 7 months ago
[meta] Improve score on CSS3Test
Nightly is now at 67% (BTW: Chrome 29 stable is at 66% but including false positives). Fixing / unifying CSS image value rendering should give us a great improvement, though there's still a lot to do. 100% seems unrealistic e. g. due to the Speech test (unless that's something Mozilla is interested in supporting in Firefox). However, a solid >90% doesn't seem too far off, does it?
I did not check yet whether the tests make sense, are up-to-date (to the specs) or if there are false negatives, but I doubt there's any error. – image-resolution – icon & content: icon – nav-* – ::value, ::choices, ::repeat-* – perspective: 0 – line-break – overflow-wrap – text-align: start end; – word-spacing & letter-spacing (extended) – hanging-punctuation – text-decoration (extended) & text-emphasis* – font-variant: none & (extended) & font-variant-ligatures: none – attr(data-*) (that one's weird) – … do not have a bug yet. Will file new ones as soon as I have time (anyone is welcome to do this, though). Bug 276079 needs a followup. `text-indent: hanging` should probably be handled by Bug 784648. Need to check if Bug 702508 covers all failing flex box tests. Left to check: Writing Modes part.
I think this shouldn't depend on metabugs like bug 631373; it should depend only on specific bugs that cause failures of these tests.
Yeah, sorry, that one should not be there.
There IS a speech bug. Yay! Also checked the Writing Modes part, bugs need to be filed for: – text-orientation – caption-side – text-combine-horizontal Now I got everything in the test at least mentioned in this bug. Time to file bugs! Later, though.
(In reply to Florian Bender from comment #1) > – font-variant: none & (extended) & font-variant-ligatures: none Ah, adventures in Webkit targeting... These are the only two flavors of the "font-variant" subproperties that Webkit supports.
(In reply to John Daggett (:jtd) from comment #6) > (In reply to Florian Bender from comment #1) > > – font-variant: none & (extended) & font-variant-ligatures: none > > Ah, adventures in Webkit targeting... These are the only two flavors of the > "font-variant" subproperties that Webkit supports. Ok, oops. The problem appears to be that these *values* were added after I wrote the initial patches for feature support. Nothing to do with Webkit... Will file a bug on these additions.
Thanks for filing! There is also a failing test for `font-variant: sub lining-nums contextual ruby;`. Is there a bug for that, too? Is that behind a flag? All other tests (except 'none' & unicode-range) succeed.
Sorry, looks like the stuff in Bug 651693 (resp. depending bugs) needs to be switched on by default, unless Bug 872301 can be fixed in time. Should I add Bug 651693 back again to the dependancy list, then? Could not find a bug for flipping the pref …
There has been a big update to css3test.com with the following changes (according to https://github.com/LeaVerou/css3test/commits/gh-pages): – update Writing Modes test – add dpcm test to image-resolution – add CSS Regions test – update CSS Text Decoration test – update CSS Text test – add Grid Layout test – fix transform/perspective test The following tests were changed (08th Sep), I'm also mentioning what the changes mean to this bug (whereby "[no change]" means no change to the previously posted lists, mostly because "sibling" tests were already failing, or new tests pass): – [no change] PASSing new direction and updated unicode-bidi properties – [no change] still FAILing caption-side & text-combine-horizontal – [no change] FAILing new image-resolution: *dpcm test – [NEW] FAILing CSS Regions (added bug) – [no change] still FAILing text-underline-position, text-emphasis* – [no change] PASSing updated text-shadow test – [no change] PASSing new word-wrap test – [no change] still FAILing parts of text-align but – [NEW] text-align: [char] test has been removed (removed dependent bug, sadly) – [no change] still FAILing text-justify – [NEW] FAILing CSS Grid (except `display: [inline-]grid;` but that's triggering XUL Grid; added bug) – [NEW] PASSing perspective-origin & transform-origin (waiting on Bug 712484 to resolve)
The test for `perspective: 0` is invalid ("Lengths must be positive.", http://www.w3.org/TR/css3-transforms/#perspective-property) and will likely be removed shortly (at least there's a pull request ready, see https://github.com/LeaVerou/css3test/pull/54), so there's one less bug to file. BTW, stock Nightly (2013-09-09) is now at 62% (including false positive for `display: (inline-)grid`) vs. Firefox 24b9 at 58% (mostly due to font magic) vs. Chrome 29 stable at 60% (including false positive for `background-repeat: space|round;`) vs. Safari 6.0.5 at 51% (but v7 will improve its score significantly, also with false positive).
Bug 914360 does not improve the score (the opposite), but it corrects it. Thus adding as dependency.
Depends on: 914360
(In reply to Florian Bender from comment #1) > – attr(data-*) (that one's weird) This is an actual bug in Gecko: Bug 435426.
Depends on: 435426
Tom's Hardware Guide's "Web Browser Grand Prix" (tracking bug 933949) includes css3test.
Latest Nightly - 630 tests out of 1039 Firefox Beta - 557 tests out of 1039 Chrome 31 - 588 tests out of 1039 IE 11 - 587 tests out of 1039
Chrome 33 - 597 tests out of 1039 Opera 19 - 613 tests out of 1039
Component: General → Tracking
QA Contact: chofmann
Latest Nightly - 667 tests out of 1338 Latest Aurora (29) - 667 tests out of 1338 Firefox Beta (28) - 589 tests out of 1338 Chrome 33 - 658 tests out of 1338 Safari 7 - 644 tests out of 1338 (minus some points for background-repeat)
Latest Nightly vs. Chrome 33 is not a fair comparison, you have to compare latest Nightly with Chrome Canary and latest stable of Chrome with latest stable of Firefox… ;) Latest Firefox: 575 tests out of 1338 Latest Chrome: 658 tests out of 1338 Latest Nightly: 667 tests out of 1338 Latest Chrome Canary: 675 tests out of 1338 But good work @firefox devs, it's a huge jump from 589 to 667!
Sure. I must confess I was a bit too lazy to download Canary atm. Thanks for the numbers Sören.
Opera Next 20 got 813.
Note that pre-release versions are likely to have a higher score than the resulting release, e.g. most font tests will fail in Fx 29 Release because they are only enabled in Nightly and Aurora (by default).
For now: 33 Nightly: 686 out of 1341, 59% 38 Canary: 709 out of 1341, 60%
I am convinced you should be careful at not reaching any hard, solid conclusion regarding Lea Verou meta-test page. For many reasons. - The page and tests need to be constantly updated: eg 'cursor: grab' and 'cursor: grabbing' are not being tested ( http://www.w3.org/TR/2015/CR-css-ui-3-20150707/#cursor ) and I have spotted other properties while editing this comment - The page tests properties from Working Drafts and not just from Candidate Recommendations only: eg Regions module - The page claim Firefox supports properties that we know are not implemented at all: widows and orphans ( bug 137367 ) - The page or tests claim Firefox does not support 'text-combine-upright' but that is not true: http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-combine-upright-parsing-valid-001.htm and bug 1097499 : no working implementation but there is support with Firefox 42 if/when layout.css.text-combine-upright.enabled is set to true - There are many false positives and false negatives in those tests and that page: Chrome supports a lot more than what the page claims due to vendor-prefixes The most important reason why this page should not be taken seriously (otherwise, at least, with a lot of salt) is that implementation is not being tested and Lea Verou is fully aware of this: "Caution: This test checks which CSS3 features the browser recognizes, not whether they are implemented correctly."
(In reply to Gérard Talbot from comment #23) > - The page and tests need to be constantly updated: eg 'cursor: grab' and > 'cursor: grabbing' are not being tested ( > http://www.w3.org/TR/2015/CR-css-ui-3-20150707/#cursor ) Filed an issue to implement those values. > and I have spotted > other properties while editing this comment Would be good if you filed issues against those on the css3test GitHub repo. > - The page tests properties from Working Drafts and not just from Candidate > Recommendations only: eg Regions module As far as I know the implementation criteria on css3test.com does not follow the status of the specifications but requires at least one working implementation of a spec. > - The page claim Firefox supports properties that we know are not > implemented at all: widows and orphans ( bug 137367 ) As the page states at the top, it only tests whether a browser recognizes a feature, not whether its implemented correctly. While 'widows' and 'orphans' are not implemented yet, Firefox already exposes them through the DOM. Non-the-less this should be mentioned on the page, so I created an issue for that. > - The page or tests claim Firefox does not support 'text-combine-upright' > but that is not true: > http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/ > text-combine-upright-parsing-valid-001.htm 'text-combine-upright' is not tested yet. Instead it still tests for 'text-combine-horizontal'. Also, Firefox *does not support* 'text-combine-upright' yet. > - There are many false positives and false negatives in those tests and that7 > page: Chrome supports a lot more than what the page claims due to > vendor-prefixes Not sure what's the rule for the page when features are behind prefixes, though you should ask Lea about that. > The most important reason why this page should not be taken seriously > (otherwise, at least, with a lot of salt) is that implementation is not > being tested and Lea Verou is fully aware of this: "Caution: This test > checks which CSS3 features the browser recognizes, not whether they are > implemented correctly." Same counts in big parts for html5test.com (which is tracked in bug 802882, btw.). Though web authors refer to those pages for comparing how good browsers implement the (upcoming) web standards. Bad scores may result in bad reputation. So it should be somewhat important to Mozilla to get a higher rank on those pages. And for implementers they provide at least a rough overview where feature implementation stands and may help to decide what to implement next. If you believe css3test.com does not offer good enough measurement in that relation, you should talk to Lea and/or file bugs against it. Sebastian  https://github.com/LeaVerou/css3test/issues/103  https://github.com/LeaVerou/css3test/issues/104  https://github.com/LeaVerou/css3test/issues/105  bug 1097499
We're still at 67% using Nightly.
You need to log in before you can comment on or make changes to this bug.