Intermittent devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0

RESOLVED FIXED in Firefox 65

Status

defect
P5
normal
RESOLVED FIXED
7 months ago
5 months ago

People

(Reporter: intermittent-bug-filer, Assigned: jdescottes)

Tracking

({intermittent-failure})

unspecified
Firefox 66
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox65 fixed, firefox66 fixed)

Details

(Whiteboard: [stockwell fixed])

Attachments

(4 attachments)

Reporter

Description

7 months ago
treeherder
Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=212705324&repo=autoland

https://queue.taskcluster.net/v1/task/dX6mq8rnTQqYFCzORNiMIA/runs/0/artifacts/public/logs/live_backing.log

21:09:47     INFO - TEST-START | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js
21:09:47     INFO - Entering test bound 
21:09:47     INFO - Adding a new tab with URL: data:text/html;charset=utf8,%0A%20%20%3Ctest-component%3E%3C/test-component%3E%0A%20%20%3Cscript%3E%0A%20%20%20%20'use%20strict';%0A%20%20%20%20customElements.define('test-component',%20class%20extends%20HTMLElement%20%7B%0A%20%20%20%20%20%20constructor()%20%7B%0A%20%20%20%20%20%20%20%20super();%0A%20%20%20%20%20%20%20%20let%20shadowRoot%20=%20this.attachShadow(%7Bmode:%20'open'%7D);%0A%20%20%20%20%20%20%20%20shadowRoot.innerHTML%20=%0A%20%20%20%20%20%20%20%20%20%20'%3Cdiv%20style=%22width:30px;height:30px;background:rgb(0,%20128,%200)%22%3E%3C/div%3E';%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D);%0A%20%20%3C/script%3E
21:09:47     INFO - Tab added and finished loading
21:09:47     INFO - Opening the inspector
21:09:47     INFO - Opening the toolbox
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_GRIDS"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Toolbox opened and focused
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Select the green node
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Selecting the node for '[Front for domnode/server1.conn83.child1/domnode39]'
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Take a screenshot of the green node and verify it looks as expected
21:09:47     INFO - Call screenshotNode() and wait until the screenshot is found in the Downloads
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Create an image using the downloaded fileas source
21:09:47     INFO - Waiting for event: 'load' on [object HTMLImageElement].
21:09:47     INFO - Got event: 'load' on [object HTMLImageElement].
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
21:09:47     INFO - TEST-INFO | started process screenshot
21:09:48     INFO - TEST-INFO | screenshot: exit 0
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:80
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - Not taking screenshot here: see the one that was previously logged
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected green component - Got 0, expected 128
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:81
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected blue component - 
21:09:48     INFO - Leaving test bound 
21:09:48     INFO - Removing tab.
21:09:48     INFO - Waiting for event: 'TabClose' on [object XULElement].
21:09:48     INFO - Got event: 'TabClose' on [object XULElement].
21:09:48     INFO - Tab removed and finished closing
21:09:48     INFO - GECKO(2644) | MEMORY STAT | vsize 902MB | vsizeMaxContiguous 379MB | residentFast 394MB | heapAllocated 137MB
21:09:48     INFO - TEST-OK | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | took 880ms
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)
Julian, could you please take a quick look at this one when you get a chance?
It's been failing ~30 times a week since it was added.
Flags: needinfo?(jdescottes)
Assignee

Comment 6

6 months ago
It looks like the previously downloaded screenshot from the iframe test gets reused here. We could try explicitly removing the downloaded file once the image has been loaded.

However I couldn't reproduce it at all on try:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8aa977cc8131a31a69955ccc3455d9b2a160cec1

I propose to try landing a fix and leave-open to see if it improves the situation.
Flags: needinfo?(jdescottes)
Assignee

Updated

6 months ago
Keywords: leave-open

Comment 7

6 months ago
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f02989fbbd6b
Delete downloaded screenshots in inspector screenshot tests;r=pbro
Comment hidden (Intermittent Failures Robot)
In the last 7 days, there have been 37 failures.

Most of the failures are on windows10-64, windows7-32.
Affected build types: opt, pgo.

An example of a recent log file: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=218450663&repo=autoland&lineNumber=9752

And the relevant part of the log:
00:06:41     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
00:06:41     INFO - TEST-INFO | started process screenshot
00:06:42     INFO - TEST-INFO | screenshot: exit 0
00:06:42     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
00:06:42     INFO - Stack trace:
00:06:42     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
00:06:42     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
00:06:42     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:focusedOrLoaded/<:803
00:06:42     INFO - Not taking screenshot here: see the one that was previously logged


:gl As you are the triage owner of this component, could you please take a look at this?
Thank you!
Flags: needinfo?(gl)
Whiteboard: [stockwell needswork]
Comment hidden (Intermittent Failures Robot)
This has 24 failures in the last 7 days, all on windows 7 pgo and opt: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2018-12-23&endday=2018-12-30&tree=trunk&bug=1508435

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=219295421&repo=mozilla-inbound&lineNumber=9754

11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
11:34:25     INFO - TEST-INFO | started process screenshot
11:34:25     INFO - TEST-INFO | screenshot: exit 0
11:34:25     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
11:34:25     INFO - Stack trace:
11:34:25     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
11:34:25     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
11:34:25     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
11:34:25     INFO - Not taking screenshot here: see the one that was previously logged


Julian since you had a crack at this in the past could please take a look over this? Thank you
Flags: needinfo?(jdescottes)
Comment hidden (Intermittent Failures Robot)
Assignee

Comment 14

6 months ago
I'll either fix or disable on windows. I think the issue is really related to windows reusing the file from the previous test. We generate the screenshot filename to look like:

  Screen Shot 2018-12-31 at 09.58.40.png

ie "Screen Shot YYYY-MM-dd at HH.mm.ss.png". When the test fails, the shadow DOM test runs in the same "second" as the iframe test. I don't understand why deleting the file after the test doesn't fix the issue, but there might be caching involved somewhere else.

I have two try pushes ongoing. One trying to move the file, if for any reason this works better than deleting the file. The other simply to wait one second between tests.
Assignee: nobody → jdescottes
Status: NEW → ASSIGNED
Flags: needinfo?(jdescottes)
Assignee

Updated

6 months ago
Flags: needinfo?(gl)
Assignee

Comment 15

6 months ago
Waiting for one second seems to fix the intermittent: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ce5eb765be430f316afa12f5f744ef3369e288a 

Compare to the default: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a632cf1f5f1edf1d6c456cf02e7a171fd084ad73 

(moving the file showed no improvements compared to the default).
Comment hidden (Intermittent Failures Robot)
Attachment #9034559 - Flags: review?(jmaher) → review+
Keywords: checkin-needed
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]

Comment 19

5 months ago
Pushed by rmaries@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/22aafd088806
Disabled devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js on Windows pgo/opt. r=jmaher
Keywords: checkin-needed
Comment hidden (Intermittent Failures Robot)

Comment 22

5 months ago
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ed6cbd03dbc2
Wait for one second between each screenshot test;r=pbro
Assignee

Comment 23

5 months ago
I just landed the patch that fixes this intermittent, can we reenable?

Comment 24

5 months ago
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ac819553ec26
Enable browser_markup_screenshot_node_shadowdom;r=jmaher
Assignee

Updated

5 months ago
Keywords: leave-open

Comment 25

5 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/ed6cbd03dbc2
https://hg.mozilla.org/mozilla-central/rev/ac819553ec26
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 66
Comment hidden (Intermittent Failures Robot)
You need to log in before you can comment on or make changes to this bug.