Intermittent /css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html

RESOLVED FIXED in Firefox 70

Status

()

defect
P5
normal
RESOLVED FIXED
2 months ago
27 days ago

People

(Reporter: intermittent-bug-filer, Assigned: dholbert)

Tracking

(Regression, {intermittent-failure, regression})

unspecified
mozilla70
Points:
---

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox68 unaffected, firefox69 wontfix, firefox70 fixed)

Details

(Whiteboard: [retriggered][stockwell disabled])

Attachments

(1 attachment)

This is a 1px difference, at the bottom-right antialiased fringe of an "a" character (which is on top of a red border in the testcase but occluded by that border in the reference case).

The "pure reftest" version of this test does have an annotation for 1px being fuzzy on Windows:
https://searchfox.org/mozilla-central/rev/1133b6716d9a8131c09754f3f29288484896b8b6/layout/reftests/w3c-css/submitted/ruby/reftest.list#3
...so this seems like a known issue. We can either copy over that fuzzy annotation into a wpt .ini file for this testcase, or (better) someone could investigate fixing this by tweaking the stacking order and/or specifying a sans-serif font perhaps.

Whiteboard: [retriggered]

Based on Comment 2, I think the right thing is for someone from dholbert's team to decide how to fix this intermittent.

Flags: needinfo?(mjzffr) → needinfo?(dholbert)

We're seeing occasional fuzzy-mismatch failures with a single pixel differing,
where an antialiased "a" serif overlaps an adjacent table border. Hopefully a
sans-serif font will reduce the likelihood of this fuzziness.

The attached patch should hopefully fix the copy of this test that lives in layout/reftests. (And I'm removing the fuzzy-if-windows annotation there, to see if it actually helps.)

Assuming it fixes the fuzziness in that copy of the test, then this bug here (with fuzzy mismatches in the WPT copy) will be addressed after this patch is landed, and its changes are synchronized to upstream wpt/css/vendor-imports, and then synchronized back into our WPT import.

Here's a Try run to test these changes to the layout/reftests copy of this test:
[1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3ef70a5d0c838c554218299c0bc1c7bdeef7e015

Flags: needinfo?(dholbert)
Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/56f5a5873ad9
Use sans-serif font in reftest ruby-inlinize-blocks-002.html to reduce likelihood of fuzziness. r=jfkthame

Triggered autoland.

I'm also adding "leave-open" because it'll probably be a week or so until this is actually fully fixed in the WPT copy of this test (via the synchronization upstream & back referred to in comment 11).

Assignee: nobody → dholbert
Status: NEW → ASSIGNED
Keywords: leave-open
Whiteboard: [retriggered][stockwell disable-recommended] → [retriggered]

We might as well annotate this test to stop the intermittent failures until the synchronization described in comment 11 has completed.

Actually, it looks like we did already annotate this test as "disabled", here:
https://hg.mozilla.org/mozilla-central/rev/c271ae1b60b9043ee090a2f6c46f3003093ce6b2#l4.3
...but the annotation is extremely specific: only on win32 / x86, with a particular version 6.1.7601 [of windows?].

I think this fuzzy failure really affected all windows builders (regardless of version/architecture). So the failure reports that have been getting through are for the jobs that aren't covered by that extremely-specific annotation -- namely, 64-bit windows testruns.

Let's just generify the annotation to make this test disabled on windows entirely, and then we can reenable it (removing the .ini file) in a week or so once the comment 11 synchronization process has completed.

Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/da96f37538d8
Temporarily disable web-platform-test ruby-inlinize-blocks-002.html on more categories of Windows testers. (no review, test-annotation-only)
Whiteboard: [retriggered][stockwell disable-recommended] → [retriggered][stockwell disabled]

Just an update: the synchronization process hasn't quite completed yet, so the .ini file can't be removed yet.

We're waiting on the next WPT-->mozilla-central sync, in order for https://searchfox.org/mozilla-central/source/testing/web-platform/tests/css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html to get a font: 16px sans-serif; declaration. After that happens, we can remove the .ini file and close out this intermittent-failure bug as fixed.

Another update, since it's been a few weeks: we're still waiting for a sufficiently-new WPT update, before we can remove the .ini file that skips this test.

We did have a recent WPT update in bug 1561193, but that one didn't update us far enough to have picked up these changes, apparently.

Just as a sanity-check -- the version of this test on github is indeed up to date, so the changes did make it that far (via comment 23's pull):
https://github.com/web-platform-tests/wpt/blob/master/css/vendor-imports/mozilla/mozilla-central-reftests/ruby/ruby-inlinize-blocks-002.html#L11

So I imagine the next WPT update will get us the fixed version of this test, and then we can reenable it by removing the .ini file.

Looks like bug 1564632 has the WPT sync that will bring in the changes that will mean we can revert comment 20 and close this out. I'll look into doing that later on today, probably.

Flags: needinfo?(dholbert)
Flags: needinfo?(dholbert)
Keywords: leave-open
Pushed by dholbert@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c8ebaec59407
part 2: re-enable web-platform-test ruby-inlinize-blocks-002.html now that it's received a fix to hopefully prevent its fuzzy mismatch. (no review, test-annotation-only)
Regressions: 1568685
Status: ASSIGNED → RESOLVED
Closed: 27 days ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
You need to log in before you can comment on or make changes to this bug.