1637887 - Intermittent Windows 10 x64 Many layout/reftest failures eg. layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html == layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html | image comparison, max difference: 255, number of differing pixel

Status: RESOLVED WORKSFORME

(Core :: Graphics, defect, P3)

Description

[Bogdan Tara[:bogdan_tara | bogdant]]

So far, we have 2 similar occurrences:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302217803&repo=autoland&lineNumber=1902
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/aLAv1hQDSlWPCNbl5ckI0g/runs/0/artifacts/public/logs/live_backing.log

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=302098440&repo=try&lineNumber=1885
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DoFLEvVWSVSWGPXbtoROag/runs/0/artifacts/public/logs/live_backing.log

[task 2020-05-14T06:42:57.128Z] 06:42:57     INFO - REFTEST TEST-START | layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html == layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html
[task 2020-05-14T06:42:57.148Z] 06:42:57     INFO - REFTEST TEST-LOAD | file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html | 0 / 30 (0%)
[task 2020-05-14T06:42:57.461Z] 06:42:57     INFO - REFTEST INFO | drawWindow flags = DRAWWINDOW_DRAW_CARET | DRAWWINDOW_DRAW_VIEW | DRAWWINDOW_USE_WIDGET_LAYERS; window size = 816,1039; test browser size = 800,1000
[task 2020-05-14T06:42:59.643Z] 06:42:59     INFO - REFTEST TEST-LOAD | file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html | 0 / 30 (0%)
[task 2020-05-14T06:42:59.653Z] 06:42:59     INFO - [GFX1-]: (gfxWindowsPlatform) Detected device reset: 1
[task 2020-05-14T06:42:59.654Z] 06:42:59     INFO - [GFX1-]: (gfxWindowsPlatform) scheduled device update.
[task 2020-05-14T06:42:59.659Z] 06:42:59     INFO - [GFX1-]: A content-only TDR is detected.
[task 2020-05-14T06:42:59.883Z] 06:42:59     INFO - REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html == layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html | image comparison, max difference: 255, number of differing pixels: 308727
[task 2020-05-14T06:42:59.884Z] 06:42:59     INFO - REFTEST   IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAA....
[task 2020-05-14T06:42:59.884Z] 06:42:59     INFO - REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAA.....
[task 2020-05-14T06:42:59.885Z] 06:42:59     INFO - REFTEST INFO | Saved log: START file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html
[task 2020-05-14T06:42:59.885Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] Using browser remote=true
[task 2020-05-14T06:42:59.886Z] 06:42:59     INFO - 
[task 2020-05-14T06:42:59.886Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering WaitForTestEnd
[task 2020-05-14T06:42:59.887Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners
[task 2020-05-14T06:42:59.887Z] 06:42:59     INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2020-05-14T06:42:59.888Z] 06:42:59     INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2020-05-14T06:42:59.888Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.889Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html
[task 2020-05-14T06:42:59.889Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
[task 2020-05-14T06:42:59.890Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for updateCanvasPending
[task 2020-05-14T06:42:59.891Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] HandlePendingTasksAfterMakeProgress updating canvas
[task 2020-05-14T06:42:59.892Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 1 rects
[task 2020-05-14T06:42:59.892Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] Rect: 0 0 800 1000
[task 2020-05-14T06:42:59.893Z] 06:42:59     INFO - REFTEST INFO | Saved log: Updating canvas for invalidation
[task 2020-05-14T06:42:59.894Z] 06:42:59     INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2020-05-14T06:42:59.894Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.895Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT
[task 2020-05-14T06:42:59.895Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: dispatching MozReftestInvalidate
[task 2020-05-14T06:42:59.896Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] AttrModifiedListener fired
[task 2020-05-14T06:42:59.896Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.896Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL
[task 2020-05-14T06:42:59.896Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.897Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_SPELL_CHECKS
[task 2020-05-14T06:42:59.897Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_APZ_FLUSH
[task 2020-05-14T06:42:59.897Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: done requesting APZ flush
[task 2020-05-14T06:42:59.897Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.898Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_APZ_FLUSH
[task 2020-05-14T06:42:59.898Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: apz-repaints-flushed fired
[task 2020-05-14T06:42:59.898Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.898Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
[task 2020-05-14T06:42:59.898Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for MozAfterPaint
[task 2020-05-14T06:42:59.899Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] HandlePendingTasksAfterMakeProgress waiting for a MozAfterPaint
[task 2020-05-14T06:42:59.899Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html
[task 2020-05-14T06:42:59.899Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] HandlePendingTasksAfterMakeProgress updating canvas
[task 2020-05-14T06:42:59.899Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 1 rects
[task 2020-05-14T06:42:59.900Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] Rect: 10 7 790 403
[task 2020-05-14T06:42:59.900Z] 06:42:59     INFO - REFTEST INFO | Saved log: Updating canvas for invalidation
[task 2020-05-14T06:42:59.900Z] 06:42:59     INFO - REFTEST INFO | Saved log: DoDrawWindow 10,7,780,396
[task 2020-05-14T06:42:59.900Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress
[task 2020-05-14T06:42:59.901Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FINISH
[task 2020-05-14T06:42:59.901Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: Completed
[task 2020-05-14T06:42:59.901Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2020-05-14T06:42:59.901Z] 06:42:59     INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2020-05-14T06:42:59.901Z] 06:42:59     INFO - REFTEST INFO | Saved log: START file:///Z:/task_1589437179/build/tests/reftest/tests/layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html
[task 2020-05-14T06:42:59.902Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2020-05-14T06:42:59.902Z] 06:42:59     INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2020-05-14T06:42:59.902Z] 06:42:59     INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2020-05-14T06:42:59.903Z] 06:42:59     INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2020-05-14T06:42:59.903Z] 06:42:59     INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2020-05-14T06:42:59.903Z] 06:42:59     INFO - REFTEST TEST-END | layout/reftests/forms/fieldset/dynamic-legend-scroll-1.html == layout/reftests/forms/fieldset/dynamic-legend-scroll-1-ref.html
...
...
...

Comment 1

[Bogdan Tara[:bogdan_tara | bogdant]]

Seeing this on Windows 10 x64 instances, followed by green jobs.

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Kats, please check if this is an in-tree graphics regression or an issue with the devices. At least one of the images doesn't render anything close to the test (e.g. checkerboards).

Comment 3

[Kartikaya Gupta (email:[e-mail])]

The logs have a lot of this:

[task 2020-05-14T06:42:59.653Z] 06:42:59     INFO - [GFX1-]: (gfxWindowsPlatform) Detected device reset: 1
[task 2020-05-14T06:42:59.654Z] 06:42:59     INFO - [GFX1-]: (gfxWindowsPlatform) scheduled device update.
[task 2020-05-14T06:42:59.659Z] 06:42:59     INFO - [GFX1-]: A content-only TDR is detected.

right before the failures, so it looks like a device issue rather than an in-tree change.

Comment 4

[Intermittent Failures Robot]

2 failures in 5488 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1

Platform breakdown:

• windows10-64: 1
• windows10-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637887&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 5

[Intermittent Failures Robot]

3 failures in 4727 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 2

Platform breakdown:

• windows10-64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637887&startday=2020-05-18&endday=2020-05-24&tree=all

Comment 7

[Intermittent Failures Robot]

2 failures in 5458 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1

Platform breakdown:

• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637887&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 8

[Intermittent Failures Robot]

3 failures in 4949 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• windows10-64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1637887&startday=2020-06-01&endday=2020-06-07&tree=all