Closed Bug 1383451 Opened 7 years ago Closed 5 years ago

Intermittent /css/CSS2/lists/list-style-021.xht | Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht

Categories

(Core :: CSS Parsing and Computation, defect, P5)

Product:
Core
Component:
CSS Parsing and Computation
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla67
Tracking Flags:
Tracking Status
firefox67 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: dholbert)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell unknown])

Component: web-platform-tests → CSS Parsing and Computation
Product: Testing → Core
Version: Version 3 → Trunk

This is just a 1px fuzzy mismatch -- in the cases I've looked at, the reference case has a single stray pixel that's rgb(247,247,247) just outside the image.

This feels like an antialiasing "fringe" that we're not quite clipping for some reason, and it's subtle enough (and intermittent enough) that I don't think it's high priority enough to care.

So, I think we should annotate this test with 1 pixel allowed to differ by up to 8 in its color channels, once we've got support for fuzzy matching (https://github.com/web-platform-tests/wpt/pull/12187 & https://bugzilla.mozilla.org/show_bug.cgi?id=1478472 )

Depends on: 1478472
Priority: -- → P5

There are 26 total failures in the last 7 days on linux32, linux64, windows10-64, windows7-32 all build types.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=231798318&repo=mozilla-inbound&lineNumber=16023

[task 2019-03-05T02:18:33.945Z] 02:18:33 INFO - TEST-START | /css/CSS2/lists/list-style-021.xht
[task 2019-03-05T02:18:33.961Z] 02:18:33 INFO - PID 4894 | 1551752313948 Marionette INFO Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht
[task 2019-03-05T02:18:33.981Z] 02:18:33 INFO - PID 4894 | JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 2384: TypeError: tabbrowser.getTabForBrowser is not a function
[task 2019-03-05T02:18:33.997Z] 02:18:33 INFO - PID 4894 | ++DOMWINDOW == 101 (0xe50e6c00) [pid = 5047] [serial = 101] [outer = 0xe79648d0]
[task 2019-03-05T02:18:34.118Z] 02:18:34 INFO - PID 4894 | JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 2384: TypeError: tabbrowser.getTabForBrowser is not a function
[task 2019-03-05T02:18:34.120Z] 02:18:34 INFO - PID 4894 | ++DOMWINDOW == 102 (0xe50e5400) [pid = 5047] [serial = 102] [outer = 0xe79648d0]
[task 2019-03-05T02:18:34.181Z] 02:18:34 INFO - PID 4894 | 1551752314176 Marionette INFO No differences allowed
[task 2019-03-05T02:18:34.182Z] 02:18:34 INFO - PID 4894 | 1551752314176 Marionette INFO Found 1 pixels different, maximum difference per channel 8
[task 2019-03-05T02:18:34.242Z] 02:18:34 INFO - TEST-UNEXPECTED-FAIL | /css/CSS2/lists/list-style-021.xht | Testing http://web-platform.test:8000/css/CSS2/lists/list-style-021.xht == http://web-platform.test:8000/css/CSS2/lists/list-style-021-ref.xht

Daniel is there an update here? are you working on this?

Flags: needinfo?(dholbert)
Whiteboard: [stockwell needswork:owner]

I wasn't working on this, but I can take a look.

Note that we still don't have support for "fuzzy annotations" in the WPT harness. So our options here are to mark the test as random (or skipped), or to adjust the reference case to avoid whatever is causing the 1px fringe noted in comment 81.

When I load the reference case locally, I see:

  • a circular bullet appear, while the image is loading.
  • then, the image appears after it's loaded.

I suspect that the 1px "fringe" here is some piece of the edge of the bullet that doesn't get invalidated correctly. (This is a bit odd because the fringe isn't where the bullet actually appears during the image load -- but I'm guessing that maybe the bullet moves + repaints just before the image gets painted, maybe [?])

So, I'll bet we could work around this by adding "list-style-type:none" to the reference case, but that might slightly defeat the point of the test (because the testcase effectively has "list-style-type:none" via a shorthand, and is seeing how it interacts with "list-style-image").

So, I feel a bit unethical making that edit to the test, given that it's a not a test that we wrote and it lives in a shared testsuite. So for now, until we can annotate tests as fuzzy, I feel like our only ethical course of action here is to just mark the test as disabled.

Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3682b31958e1
Skip web-platform-test list-style-021.xht since its reference occasionally renders with 1 fuzzy pixel. (no review, test annotation only)

Unless I messed something up, comment 90 should disable the test, so we won't get any more reports here (after it's merged to the various other integration branches).

Flags: needinfo?(dholbert)

https://hg.mozilla.org/mozilla-central/rev/3682b31958e1

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox67: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
Assignee: nobody → dholbert

(In reply to Daniel Holbert [:dholbert] from comment #87)

I wasn't working on this, but I can take a look.

Thank you for working on this!

Blocks: 1542002
You need to log in before you can comment on or make changes to this bug.