Bug 1498357 (wpt.fyi-firefox-fails)

[meta] Tracking bug for web platform tests that only fail in Firefox




9 months ago
2 months ago


(Reporter: bzbarsky, Unassigned)


(Depends on 25 bugs, {meta})

Dependency tree / graph

Firefox Tracking Flags

(firefox64 affected)



See https://groups.google.com/forum/#!topic/mozilla.dev.platform/LMnQq-yqbxg for a list.  We should file specific bugs blocking this one for specific failures.

One obvious issue is streams, of course.
Keywords: meta
Priority: -- → P3
Summary: Tracking bug for web platform tests that only fail in Firefox → [meta] Tracking bug for web platform tests that only fail in Firefox
Depends on: 332175
Depends on: 1328138
Depends on: 1498445
Depends on: 1425874
Depends on: 1398354
Depends on: 1499547
Depends on: 1499556
Depends on: 1499567
Depends on: 1499571
Alias: wpt.fyi-firefox-fails
Depends on: 1500492
Filed https://github.com/web-platform-tests/wpt/issues/13626 for a failure due to a test with buggy assumptions ( flex-flexitem-percentage-prescation.html )
Depends on: 1500495
Depends on: 1500590
Depends on: 1360715
Depends on: 1469045
No longer depends on: 1469045
Depends on: 1500623
Depends on: 1120171
Depends on: 1500631
Depends on: 1501023
Depends on: 1501508
Depends on: 1501057
Depends on: 1502168
Given the recent news [1] it won't be as relevant to consider the status of EdgeHTML for prioritization in other engines. Given that and the unreliable results, I've updated my script to consider only Chrome, Firefox and Safari. I also the reports auto-updating on a daily basis:


[1] https://github.com/MicrosoftEdge/MSEdge/blob/master/README.md
Depends on: 1514227
Depends on: 1514230
Depends on: 1514234
Depends on: 1514237
Depends on: 1514243
Depends on: 1514264
Depends on: 1514265
Depends on: 1514266
Depends on: 1514267
Depends on: 1514269
Depends on: 1514271
Depends on: 1514276
Depends on: 1514301
Depends on: 1514362
Depends on: 1514367
Depends on: 1514371
Depends on: 1514375
Depends on: 1514376
Depends on: 1514396
Depends on: 1514397
Depends on: 1514399
Depends on: 1514401
Depends on: 1514486
Depends on: 144795

wpt.fyi now supports a structured-query syntax that supports searching by specific statuses, per-browser. Using this feature, a search for

!firefox:pass !firefox:ok (chrome:pass|chrome:ok) (edge:pass|edge:ok) (safari:pass|safari:ok)

will show "Firefox anomalies" / tests that only fail in firefox.


Depends on: 1489308
Depends on: 1522940
Depends on: 1522963
Depends on: 1450027
Depends on: 1527874
Depends on: 1527875
Depends on: 1528689
Depends on: 1528692
Depends on: 1528706
Depends on: 1529019
Depends on: 1529051
Depends on: 1529054

https://jgraham.github.io/wptdash/ can be used to get a list of tests that fail in Firefox but pass in Chrome & Safari, based on wpt.fyi data (url subject ot change, suggestions for improvement welcome).

Depends on: 1529087
Depends on: 1529093
Depends on: 1529096
No longer depends on: 1529096
No longer depends on: 1514486
No longer depends on: 1527874
No longer depends on: 1527875
No longer depends on: 1529087
Depends on: 1529299
No longer depends on: 1514397
Depends on: 1529543

lukebjerring, is the wpt.fyi URL in comment #3 correct? In many of the columns I see "N / N" for Firefox, which seems weird.

Are the exclamation marks just in the wrong place? The insights tab on staging.wpt.fyi suggests "(firefox:!pass&firefox:!ok) (chrome:pass|chrome:ok) (edge:pass|edge:ok) (safari:pass|safari:ok)" as the query, but I ran into this bug:

Depends on: 25291
Depends on: 1533013
Depends on: 1539469
Depends on: 1543267
Depends on: 1092007
Depends on: 1545051
Depends on: 1546387
Depends on: 1546389
No longer depends on: 1546387
No longer depends on: 1529051
No longer depends on: 1514376
No longer depends on: 1514265
No longer depends on: 1529093
No longer depends on: 1514264
No longer depends on: 1500492
No longer depends on: 1514276
No longer depends on: 1514301
No longer depends on: 1514266
No longer depends on: 1514271
No longer depends on: 1514237
No longer depends on: 1500495
No longer depends on: 1514267
No longer depends on: 1514243
No longer depends on: 1514269
No longer depends on: 1514375
No longer depends on: 1529054
No longer depends on: 1514362
No longer depends on: 1501508
No longer depends on: 1543267
No longer depends on: 1539469
No longer depends on: 1514367
Depends on: 1549560
You need to log in before you can comment on or make changes to this bug.