Closed Bug 1338039 Opened 7 years ago Closed 6 years ago

Intermittent w3c-css/submitted/ui3/box-sizing-replaced-001.xht == w3c-css/submitted/ui3/box-sizing-replaced-001-ref.xht | image comparison, max difference: 255, number of differing pixels: 10070

Categories

(Core :: Layout, defect)

Product:
Core
Component:
Layout
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla60
Tracking Flags:
Tracking Status
firefox52 --- wontfix
firefox53 --- wontfix
firefox54 --- wontfix
firefox60 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: neerja)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

screenshot of reftest-analyzer showing the failure
9.87 KB, image/png
Details
Bug 1338039 - Add extremely permissive fuzzy annotation for box-sizing-replaced-001.xht to prevent failures due to paint delay reduction.
59 bytes, text/x-review-board-request
dholbert
: review+
Details 
Any chance you'd be interested in looking at this, Neerja?
Blocks: 1321707
status-firefox52: --- → affected
status-firefox53: --- → affected
status-firefox54: --- → affected
Flags: needinfo?(npancholi)
Sure, this looks similar to Bug 1313772.
I'll take a look.
Flags: needinfo?(npancholi)
Assignee: nobody → npancholi
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix
For reference: in the latest failure (from 23 Aug 2017), the third image in the first and second row is mostly unpainted. Only its very right edge gets painted, for some reason.

The 2nd-latest failure (from 15 Aug 2017) was similar, except that the first row was fine in that one -- only the second row (and that row's third image specifically) are broken there.
It looks like this test was duplicated from the W3C repo (here: Bug 1295466) and a fuzzy annotation was added about a year ago to suppress a similar failure. But now the fuzz value has increased so we are seeing this failure again.
After updating the fuzz value, this bug can be closed because the final fix for this is being tracked in Bug 1316772.
(In reply to Neerja Pancholi[:neerja] from comment #18)
> It looks like this test was duplicated from the W3C repo (here: Bug 1295466)
> and a fuzzy annotation was added about a year ago to suppress a similar
> failure.
> [...] After updating the fuzz value, this bug can be closed because the final
> fix for this is being tracked in Bug 1316772.

Hmm... I think the idea with that duplication was to allow us to have a local copy so we could use the nonstandard "MozReftestInvalidate" event to avoid the failure (which just stems from a race condition between pageload & first paint & the reftest snapshot).  MozReftestInvalidate was supposed to let us avoid the race condition by waiting until after the first paint.  But apparently it's not perfectly addressing the race condition (hence this bug), so something a bit mysterious is going on.  And I think that means the theory laid out in Bug 1316772 isn't entirely complete.

Fortunately, it looks like is a very low-frequency failure (zero failures in the past month, and 12 total failures over the 4 months before that). Here's a graph over that time period (October to today):
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1338039&startday=2017-10-01&endday=2018-03-07&tree=all

Given that low level of failures (and the fact that I'm not so sure this failure is identical to the failure in the duplicate copy), I don't think we should add the extremely permissive fuzzy annotation here.  The overhead caused by starring this bug is a rounding error for our sheriff's work at this point, so unless this regresses & gets spammy enough to draw attention, I'd lean towards leaving this as-is.
(The reason I was OK with adding an extremely permissive annotation in Bug 1295466 was *because* we have this less-permissive modified-to-be-less-failure-prone duplicate copy of the test in the tree.  So: if we copied the extremely permissive annotation, there'd be less justification for this copy to exist, I think.)
Comment on attachment 8956974 [details]
Bug 1338039 - Add extremely permissive fuzzy annotation for box-sizing-replaced-001.xht to prevent failures due to paint delay reduction.

https://reviewboard.mozilla.org/r/225922/#review231824
Attachment #8956974 - Flags: review?(dholbert) → review-
So: I think we can unassign this and just treat it as a "gathering data on a fortunately-low-frequency mysterious intermittent failure" type of bug for now, with the real work to be ultimately done in Bug 1316772.
Assignee: npancholi → nobody
oh, sorry - in comment 19 and 20, I was mistakenly thinking this was in our local copy of the w3c-submitted testcase.  But this is in our (known-failing) w3c-submitted copy.

Given that -- and given that we've still got our stricter local copy -- let's take this after all.
Comment on attachment 8956974 [details]
Bug 1338039 - Add extremely permissive fuzzy annotation for box-sizing-replaced-001.xht to prevent failures due to paint delay reduction.

https://reviewboard.mozilla.org/r/225922/#review231834

::: commit-message-bccdc:1
(Diff revision 1)
> +Bug 1338039 - Add extremely permissive fuzzy annotation for box-sizing-replaced-001.xht to prevent failures due to paint delay reduction. r?dholbert
> +

Would you mind adding a second line to the commit message here, to note that we've got a stricter modified version of this test elsewhere in the tree? (That makes this extremely-permissive annotation less bad.)

r=me with that
Attachment #8956974 - Flags: review- → review+
Assignee: nobody → npancholi
(In reply to Daniel Holbert [:dholbert] from comment #24)
> Would you mind adding a second line to the commit message here, to note that
> we've got a stricter modified version of this test elsewhere in the tree?
> (That makes this extremely-permissive annotation less bad.)

Thanks Daniel! :)

To clarify comment 18, the test that we are updating the fuzzy annotation for is the W3C copy. There is an *existing* local copy of this test that was created as part of Bug 1295466 (copy of test is here->https://searchfox.org/mozilla-central/source/layout/reftests/bugs/1295466-1.xhtml) and this copy is the stricter modified version which makes it ok for us to have this extremely permissive annotation on the W3C copy.
Pushed by npancholi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b9d430803080
Add extremely permissive fuzzy annotation for box-sizing-replaced-001.xht to prevent failures due to paint delay reduction. r=dholbert
https://hg.mozilla.org/mozilla-central/rev/b9d430803080
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox60: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: