Closed Bug 1481664 Opened 3 years ago Closed 3 years ago

Intermittent TEST-UNEXPECTED-PASS | bugs/501627-1.html == bugs/501627-1-ref.html | image comparison, max difference: 2, number of differing pixels: 39042

Categories

(Core :: Graphics: WebRender, defect, P3)

Product:
Core
Component:
Graphics: WebRender
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla63
Tracking Flags:
Tracking Status
firefox63 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: aiakab)

References

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell fixed:other])

Attachments

(1 file, 1 obsolete file)

Adjusted the fuzz-if range
824 bytes, patch
jmaher
: review+
Details | Diff | Splinter Review 
Filed by: cbrindusan [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=192638058&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/VYgQgsLOTyuPhSCjrulgvg/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/VYgQgsLOTyuPhSCjrulgvg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1&only_show_unexpected=1

14:15:24     INFO - REFTEST TEST-START | file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1.html == file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1-ref.html
14:15:24     INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1.html | 1371 / 2041 (67%)
14:15:24     INFO - ++DOMWINDOW == 71 (0x11d682800) [pid = 1980] [serial = 3882] [outer = 0x115d40800]
14:15:24     INFO - REFTEST TEST-LOAD | file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1-ref.html | 1371 / 2041 (67%)
14:15:24     INFO - ++DOMWINDOW == 72 (0x11d682c00) [pid = 1980] [serial = 3883] [outer = 0x115d40800]
14:15:24     INFO - REFTEST INFO | REFTEST fuzzy test (5, 83244) <= (2, 39042) <= (6, 97456)
14:15:25     INFO - REFTEST TEST-UNEXPECTED-PASS | file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1.html == file:///Users/cltbld/tasks/task_1533675836/build/tests/reftest/tests/layout/reftests/bugs/501627-1-ref.html | image comparison, max difference: 2, number of differing pixels: 39042
There have been 138 failures in the last 7 days.
All of the failures occurr on macosx64-qr / debug.
This was filed just 4 days ago.

Recent log file:
https://treeherder.mozilla.org/logviewer.html#?job_id=193477464&repo=mozilla-central&lineNumber=102738

Reftest analyzer:
https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/ZATJAvHeT-yzTEWsbM4mnw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1&only_show_unexpected=1

:svoisen would you please take a look?
Flags: needinfo?(svoisen)
Whiteboard: [stockwell needswork]
This looks like a regression from:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=62e129cc3495
(and somebody else did a bunch of retriggers and didn't comment here).

See:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&tochange=62e129cc34958bdfa55e8254b9a257737258686a&fromchange=f3022e524fc7&filter-searchStr=macosx64-qr%20reftest-e10s-2

However, it doesn't seem to have happened much beyond a few pushes after the original push.

bug 1470638 and bug 1482211 are unexpected passes of the same test.
Blocks: 1480433
Component: Layout → Graphics: WebRender
Priority: P5 → P3
See Also: → 1470638, 1482211
(The fuzzy-if() range in the manifest should just be adjusted downwards to accomodate this, probably.)
Attached patch Disabled on mac (obsolete) — — Splinter Review 
Disabled this reftest if needed
Assignee: nobody → aiakab
Attachment #9002307 - Flags: review?(jmaher)
Comment on attachment 9002307 [details] [diff] [review]
Disabled on mac

Review of attachment 9002307 [details] [diff] [review]:
-----------------------------------------------------------------

::: layout/reftests/bugs/reftest.list
@@ +1389,4 @@
>  == 501257-1a.html 501257-1-ref.html
>  == 501257-1b.html 501257-1-ref.html
>  == 501257-1.xhtml 501257-1-ref.xhtml
> +fuzzy-if(webrender,5-6,83244-97456) skip-if(cocoaWidget) == 501627-1.html 501627-1-ref.html # Bug 1481664

We should just adjust the fuzz range to make this pass
Comment on attachment 9002307 [details] [diff] [review]
Disabled on mac

Review of attachment 9002307 [details] [diff] [review]:
-----------------------------------------------------------------

I agree, lets adjust the fuzzy-if

::: layout/reftests/bugs/reftest.list
@@ +1389,4 @@
>  == 501257-1a.html 501257-1-ref.html
>  == 501257-1b.html 501257-1-ref.html
>  == 501257-1.xhtml 501257-1-ref.xhtml
> +fuzzy-if(webrender,5-6,83244-97456) skip-if(cocoaWidget) == 501627-1.html 501627-1-ref.html # Bug 1481664

if we do the fuzz, it would be:
fuzzy-if(webrender&&cocoaWidget,2,39042)

this would conflict with the earlier one, so we would need to edit the existing fuzzy-if to be:
fuzzy-if(webrender,2-6,39042-97456)
Attachment #9002307 - Flags: review?(jmaher) → review-
Eliza, could you look into adjusting the fuzzy-if for this test?
Flags: needinfo?(ebalazs)
Thank you for jumping on this Joel.
Flags: needinfo?(svoisen)
Attachment #9002307 - Attachment is obsolete: true
Flags: needinfo?(ebalazs)
Attachment #9002676 - Flags: review?(jmaher)
Comment on attachment 9002676 [details] [diff] [review]
Adjusted the fuzz-if range

Review of attachment 9002676 [details] [diff] [review]:
-----------------------------------------------------------------

looks great
Attachment #9002676 - Flags: review?(jmaher) → review+
Keywords: checkin-needed
Whiteboard: [stockwell disable-recommended] → [stockwell fixed:other]
Pushed by dluca@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7ab2f93f6a46
Adjusted the fuzzy-if for 501627-1.html. r=jmaher
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/7ab2f93f6a46
Status: NEW → RESOLVED
Closed: 3 years ago
status-firefox63: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
You need to log in before you can comment on or make changes to this bug.