Closed Bug 1508435 Opened 6 years ago Closed 5 years ago

Intermittent devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0

Categories

(DevTools :: Inspector, defect, P5)

Product:
DevTools
Component:
Inspector
Type:
defect

Tracking

(firefox65 fixed, firefox66 fixed)

Status:
RESOLVED FIXED
Milestone:
Firefox 66
Tracking Flags:
Tracking Status
firefox65 --- fixed
firefox66 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: jdescottes)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell fixed])

Attachments

(4 files)

Bug 1508435 - Delete downloaded screenshots in inspector screenshot tests;r=pbro
47 bytes, text/x-phabricator-request
Details | Review
Bug 1508435 - Wait for one second between each screenshot test;r=pbro
47 bytes, text/x-phabricator-request
Details | Review
Disabled test on Windows pgo/opt
831 bytes, patch
jmaher
: review+
Details | Diff | Splinter Review
Bug 1508435 - Enable browser_markup_screenshot_node_shadowdom;r=jmaher
47 bytes, text/x-phabricator-request
Details | Review 
Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=212705324&repo=autoland

https://queue.taskcluster.net/v1/task/dX6mq8rnTQqYFCzORNiMIA/runs/0/artifacts/public/logs/live_backing.log

21:09:47     INFO - TEST-START | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js
21:09:47     INFO - Entering test bound 
21:09:47     INFO - Adding a new tab with URL: data:text/html;charset=utf8,%0A%20%20%3Ctest-component%3E%3C/test-component%3E%0A%20%20%3Cscript%3E%0A%20%20%20%20'use%20strict';%0A%20%20%20%20customElements.define('test-component',%20class%20extends%20HTMLElement%20%7B%0A%20%20%20%20%20%20constructor()%20%7B%0A%20%20%20%20%20%20%20%20super();%0A%20%20%20%20%20%20%20%20let%20shadowRoot%20=%20this.attachShadow(%7Bmode:%20'open'%7D);%0A%20%20%20%20%20%20%20%20shadowRoot.innerHTML%20=%0A%20%20%20%20%20%20%20%20%20%20'%3Cdiv%20style=%22width:30px;height:30px;background:rgb(0,%20128,%200)%22%3E%3C/div%3E';%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D);%0A%20%20%3C/script%3E
21:09:47     INFO - Tab added and finished loading
21:09:47     INFO - Opening the inspector
21:09:47     INFO - Opening the toolbox
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_GRIDS"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Toolbox opened and focused
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Select the green node
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Selecting the node for '[Front for domnode/server1.conn83.child1/domnode39]'
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Take a screenshot of the green node and verify it looks as expected
21:09:47     INFO - Call screenshotNode() and wait until the screenshot is found in the Downloads
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Create an image using the downloaded fileas source
21:09:47     INFO - Waiting for event: 'load' on [object HTMLImageElement].
21:09:47     INFO - Got event: 'load' on [object HTMLImageElement].
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
21:09:47     INFO - TEST-INFO | started process screenshot
21:09:48     INFO - TEST-INFO | screenshot: exit 0
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:80
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - Not taking screenshot here: see the one that was previously logged
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected green component - Got 0, expected 128
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:81
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected blue component - 
21:09:48     INFO - Leaving test bound 
21:09:48     INFO - Removing tab.
21:09:48     INFO - Waiting for event: 'TabClose' on [object XULElement].
21:09:48     INFO - Got event: 'TabClose' on [object XULElement].
21:09:48     INFO - Tab removed and finished closing
21:09:48     INFO - GECKO(2644) | MEMORY STAT | vsize 902MB | vsizeMaxContiguous 379MB | residentFast 394MB | heapAllocated 137MB
21:09:48     INFO - TEST-OK | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | took 880ms
Julian, could you please take a quick look at this one when you get a chance?
It's been failing ~30 times a week since it was added.
Flags: needinfo?(jdescottes)
It looks like the previously downloaded screenshot from the iframe test gets reused here. We could try explicitly removing the downloaded file once the image has been loaded.

However I couldn't reproduce it at all on try:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8aa977cc8131a31a69955ccc3455d9b2a160cec1

I propose to try landing a fix and leave-open to see if it improves the situation.
Flags: needinfo?(jdescottes)
Keywords: leave-open
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f02989fbbd6b
Delete downloaded screenshots in inspector screenshot tests;r=pbro
In the last 7 days, there have been 37 failures.

Most of the failures are on windows10-64, windows7-32.
Affected build types: opt, pgo.

An example of a recent log file: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=218450663&repo=autoland&lineNumber=9752

And the relevant part of the log:
00:06:41     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
00:06:41     INFO - TEST-INFO | started process screenshot
00:06:42     INFO - TEST-INFO | screenshot: exit 0
00:06:42     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
00:06:42     INFO - Stack trace:
00:06:42     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
00:06:42     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
00:06:42     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:focusedOrLoaded/<:803
00:06:42     INFO - Not taking screenshot here: see the one that was previously logged


:gl As you are the triage owner of this component, could you please take a look at this?
Thank you!
Flags: needinfo?(gl)
Whiteboard: [stockwell needswork]
This has 24 failures in the last 7 days, all on windows 7 pgo and opt: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2018-12-23&endday=2018-12-30&tree=trunk&bug=1508435

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=219295421&repo=mozilla-inbound&lineNumber=9754

11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
11:34:25     INFO - TEST-INFO | started process screenshot
11:34:25     INFO - TEST-INFO | screenshot: exit 0
11:34:25     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
11:34:25     INFO - Stack trace:
11:34:25     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
11:34:25     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
11:34:25     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
11:34:25     INFO - Not taking screenshot here: see the one that was previously logged


Julian since you had a crack at this in the past could please take a look over this? Thank you
Flags: needinfo?(jdescottes)
I'll either fix or disable on windows. I think the issue is really related to windows reusing the file from the previous test. We generate the screenshot filename to look like:

  Screen Shot 2018-12-31 at 09.58.40.png

ie "Screen Shot YYYY-MM-dd at HH.mm.ss.png". When the test fails, the shadow DOM test runs in the same "second" as the iframe test. I don't understand why deleting the file after the test doesn't fix the issue, but there might be caching involved somewhere else.

I have two try pushes ongoing. One trying to move the file, if for any reason this works better than deleting the file. The other simply to wait one second between tests.
Assignee: nobody → jdescottes
Status: NEW → ASSIGNED
Flags: needinfo?(jdescottes)
Flags: needinfo?(gl)
Waiting for one second seems to fix the intermittent: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ce5eb765be430f316afa12f5f744ef3369e288a 

Compare to the default: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a632cf1f5f1edf1d6c456cf02e7a171fd084ad73 

(moving the file showed no improvements compared to the default).
Attachment #9034559 - Flags: review?(jmaher) → review+
Keywords: checkin-needed
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Pushed by rmaries@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/22aafd088806
Disabled devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js on Windows pgo/opt. r=jmaher
Keywords: checkin-needed
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ed6cbd03dbc2
Wait for one second between each screenshot test;r=pbro
I just landed the patch that fixes this intermittent, can we reenable?
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ac819553ec26
Enable browser_markup_screenshot_node_shadowdom;r=jmaher
Keywords: leave-open
https://hg.mozilla.org/mozilla-central/rev/ed6cbd03dbc2
https://hg.mozilla.org/mozilla-central/rev/ac819553ec26
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
status-firefox66: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 66
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: