1508435 - Intermittent devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0

Status: RESOLVED FIXED

(DevTools :: Inspector, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=212705324&repo=autoland

https://queue.taskcluster.net/v1/task/dX6mq8rnTQqYFCzORNiMIA/runs/0/artifacts/public/logs/live_backing.log

21:09:47     INFO - TEST-START | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js
21:09:47     INFO - Entering test bound 
21:09:47     INFO - Adding a new tab with URL: data:text/html;charset=utf8,%0A%20%20%3Ctest-component%3E%3C/test-component%3E%0A%20%20%3Cscript%3E%0A%20%20%20%20'use%20strict';%0A%20%20%20%20customElements.define('test-component',%20class%20extends%20HTMLElement%20%7B%0A%20%20%20%20%20%20constructor()%20%7B%0A%20%20%20%20%20%20%20%20super();%0A%20%20%20%20%20%20%20%20let%20shadowRoot%20=%20this.attachShadow(%7Bmode:%20'open'%7D);%0A%20%20%20%20%20%20%20%20shadowRoot.innerHTML%20=%0A%20%20%20%20%20%20%20%20%20%20'%3Cdiv%20style=%22width:30px;height:30px;background:rgb(0,%20128,%200)%22%3E%3C/div%3E';%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D);%0A%20%20%3C/script%3E
21:09:47     INFO - Tab added and finished loading
21:09:47     INFO - Opening the inspector
21:09:47     INFO - Opening the toolbox
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_GRIDS"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Toolbox opened and focused
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Select the green node
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Selecting the node for '[Front for domnode/server1.conn83.child1/domnode39]'
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "CLEAR_FLEXBOX"
21:09:47     INFO - Take a screenshot of the green node and verify it looks as expected
21:09:47     INFO - Call screenshotNode() and wait until the screenshot is found in the Downloads
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_OFFSET_PARENT"
21:09:47     INFO - GECKO(2644) | console.log: "[DISPATCH] action type:" "UPDATE_LAYOUT"
21:09:47     INFO - Create an image using the downloaded fileas source
21:09:47     INFO - Waiting for event: 'load' on [object HTMLImageElement].
21:09:47     INFO - Got event: 'load' on [object HTMLImageElement].
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
21:09:47     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
21:09:47     INFO - TEST-INFO | started process screenshot
21:09:48     INFO - TEST-INFO | screenshot: exit 0
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:80
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - Not taking screenshot here: see the one that was previously logged
21:09:48     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected green component - Got 0, expected 128
21:09:48     INFO - Stack trace:
21:09:48     INFO - chrome://mochikit/content/browser-test.js:test_is:1303
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:81
21:09:48     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1093
21:09:48     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1084
21:09:48     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:982
21:09:48     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
21:09:48     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected blue component - 
21:09:48     INFO - Leaving test bound 
21:09:48     INFO - Removing tab.
21:09:48     INFO - Waiting for event: 'TabClose' on [object XULElement].
21:09:48     INFO - Got event: 'TabClose' on [object XULElement].
21:09:48     INFO - Tab removed and finished closing
21:09:48     INFO - GECKO(2644) | MEMORY STAT | vsize 902MB | vsizeMaxContiguous 379MB | residentFast 394MB | heapAllocated 137MB
21:09:48     INFO - TEST-OK | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | took 880ms

Comment 1

[Intermittent Failures Robot]

21 failures in 2387 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 3
* autoland: 13
* try: 2
* mozilla-central: 3

Platform breakdown:
* windows7-32: 18
* windows7-32-devedition: 2
* windows7-32-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-11-19&endday=2018-11-25&tree=all

Comment 2

[Intermittent Failures Robot]

39 failures in 2808 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 12
* autoland: 21
* try: 4
* mozilla-central: 2

Platform breakdown:
* windows10-64-devedition: 1
* windows10-64-nightly: 1
* windows10-64: 4
* windows7-32: 31
* windows7-32-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-11-26&endday=2018-12-02&tree=all

Comment 3

[Intermittent Failures Robot]

34 failures in 1862 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #30 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* try: 1
* mozilla-central: 5
* mozilla-beta: 2
* mozilla-inbound: 7
* autoland: 19

Platform breakdown:
* windows7-32-nightly: 2
* windows10-64: 4
* windows10-64-nightly: 1
* windows7-32: 25
* windows7-32-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-12-03&endday=2018-12-09&tree=all

Comment 4

[Patrick Brosset <:pbro>]

Julian, could you please take a quick look at this one when you get a chance?
It's been failing ~30 times a week since it was added.

Comment 5

[Julian Descottes [:jdescottes]]

Attachment: Bug 1508435 - Delete downloaded screenshots in inspector screenshot tests;r=pbro

Comment 6

[Julian Descottes [:jdescottes]]

It looks like the previously downloaded screenshot from the iframe test gets reused here. We could try explicitly removing the downloaded file once the image has been loaded.

However I couldn't reproduce it at all on try:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8aa977cc8131a31a69955ccc3455d9b2a160cec1

I propose to try landing a fix and leave-open to see if it improves the situation.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f02989fbbd6b
Delete downloaded screenshots in inspector screenshot tests;r=pbro

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f02989fbbd6b

Comment 9

[Intermittent Failures Robot]

30 failures in 2485 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* try: 8
* mozilla-beta: 2
* autoland: 12
* mozilla-inbound: 4
* mozilla-central: 4

Platform breakdown:
* windows10-64: 2
* windows7-32: 18
* windows7-32-devedition: 3
* windows10-64-nightly: 2
* windows7-32-nightly: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-12-10&endday=2018-12-16&tree=all

Comment 10

[Cristina Coroiu [:ccoroiu]]

In the last 7 days, there have been 37 failures.

Most of the failures are on windows10-64, windows7-32.
Affected build types: opt, pgo.

An example of a recent log file: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=218450663&repo=autoland&lineNumber=9752

And the relevant part of the log:
00:06:41     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
00:06:41     INFO - TEST-INFO | started process screenshot
00:06:42     INFO - TEST-INFO | screenshot: exit 0
00:06:42     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
00:06:42     INFO - Stack trace:
00:06:42     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
00:06:42     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
00:06:42     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
00:06:42     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
00:06:42     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:focusedOrLoaded/<:803
00:06:42     INFO - Not taking screenshot here: see the one that was previously logged


:gl As you are the triage owner of this component, could you please take a look at this?
Thank you!

Comment 11

[Intermittent Failures Robot]

52 failures in 2648 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #29 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-beta: 9
* mozilla-central: 11
* try: 4
* mozilla-inbound: 15
* autoland: 13

Platform breakdown:
* windows10-64-devedition: 4
* windows7-32-nightly: 8
* windows7-32-devedition: 3
* windows10-64-nightly: 2
* windows10-64: 8
* windows7-32: 27

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-12-17&endday=2018-12-23&tree=all

Comment 12

[Cosmin Sabou [:CosminS]]

This has 24 failures in the last 7 days, all on windows 7 pgo and opt: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2018-12-23&endday=2018-12-30&tree=trunk&bug=1508435

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=219295421&repo=mozilla-inbound&lineNumber=9754

11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected width - 
11:34:25     INFO - TEST-PASS | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected height - 
11:34:25     INFO - TEST-INFO | started process screenshot
11:34:25     INFO - TEST-INFO | screenshot: exit 0
11:34:25     INFO - TEST-UNEXPECTED-FAIL | devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js | node screenshot has the expected red component - Got 255, expected 0
11:34:25     INFO - Stack trace:
11:34:25     INFO - chrome://mochikit/content/browser-test.js:test_is:1316
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/helper_screenshot_node.js:assertSingleColorScreenshotImage:83
11:34:25     INFO - chrome://mochitests/content/browser/devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js:null:33
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest/<:1106
11:34:25     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1097
11:34:25     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:995
11:34:25     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/waitForFocusInner/focusedOrLoaded/<:803
11:34:25     INFO - Not taking screenshot here: see the one that was previously logged


Julian since you had a crack at this in the past could please take a look over this? Thank you

Comment 13

[Intermittent Failures Robot]

39 failures in 1008 pushes (0.039 failures/push) were associated with this bug in the last 7 days.

This is the #9 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 13
* autoland: 9
* try: 9
* mozilla-central: 5
* mozilla-beta: 3

Platform breakdown:
* windows10-64: 3
* windows7-32-nightly: 4
* windows10-64-devedition: 1
* windows7-32: 26
* windows10-64-nightly: 3
* windows7-32-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-12-24&endday=2018-12-30&tree=all

Comment 14

[Julian Descottes [:jdescottes]]

I'll either fix or disable on windows. I think the issue is really related to windows reusing the file from the previous test. We generate the screenshot filename to look like:

  Screen Shot 2018-12-31 at 09.58.40.png

ie "Screen Shot YYYY-MM-dd at HH.mm.ss.png". When the test fails, the shadow DOM test runs in the same "second" as the iframe test. I don't understand why deleting the file after the test doesn't fix the issue, but there might be caching involved somewhere else.

I have two try pushes ongoing. One trying to move the file, if for any reason this works better than deleting the file. The other simply to wait one second between tests.

Comment 15

[Julian Descottes [:jdescottes]]

Waiting for one second seems to fix the intermittent: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ce5eb765be430f316afa12f5f744ef3369e288a 

Compare to the default: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a632cf1f5f1edf1d6c456cf02e7a171fd084ad73 

(moving the file showed no improvements compared to the default).

Comment 16

[Julian Descottes [:jdescottes]]

Attachment: Bug 1508435 - Wait for one second between each screenshot test;r=pbro

Comment 17

[Intermittent Failures Robot]

17 failures in 421 pushes (0.04 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* try: 3
* mozilla-inbound: 5
* mozilla-central: 4
* autoland: 5

Platform breakdown:
* windows10-64: 1
* windows7-32-nightly: 2
* windows7-32: 12
* windows7-32-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2019-01-04&endday=2019-01-04&tree=all

Comment 18

[Natalia Csoregi [:nataliaCs]]

Attachment: Disabled test on Windows pgo/opt

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/22aafd088806
Disabled devtools/client/inspector/markup/test/browser_markup_screenshot_node_shadowdom.js on Windows pgo/opt. r=jmaher

Comment 20

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/22aafd088806

Comment 21

[Intermittent Failures Robot]

64 failures in 1668 pushes (0.038 failures/push) were associated with this bug in the last 7 days.

This is the #12 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* try: 11
* mozilla-inbound: 19
* autoland: 20
* mozilla-beta: 4
* mozilla-central: 10

Platform breakdown:
* windows7-32: 37
* windows10-64: 12
* windows10-64-devedition: 1
* windows7-32-nightly: 9
* windows7-32-devedition: 4
* windows10-64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2018-12-31&endday=2019-01-06&tree=all

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ed6cbd03dbc2
Wait for one second between each screenshot test;r=pbro

Comment 23

[Julian Descottes [:jdescottes]]

Attachment: Bug 1508435 - Enable browser_markup_screenshot_node_shadowdom;r=jmaher

I just landed the patch that fixes this intermittent, can we reenable?

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ac819553ec26
Enable browser_markup_screenshot_node_shadowdom;r=jmaher

Comment 25

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/ed6cbd03dbc2
https://hg.mozilla.org/mozilla-central/rev/ac819553ec26

Comment 26

[Mike de Boer [:mikedeboer]]

https://hg.mozilla.org/projects/cedar/rev/ed6cbd03dbc20866e472a587cf3275d78b01adcc
Bug 1508435 - Wait for one second between each screenshot test;r=pbro

https://hg.mozilla.org/projects/cedar/rev/ac819553ec263f7d13e67ac720bde50bdbca6887
Bug 1508435 - Enable browser_markup_screenshot_node_shadowdom;r=jmaher

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/05fec0e0ad57
https://hg.mozilla.org/releases/mozilla-beta/rev/3b5f78d0f03e

Comment 28

[Intermittent Failures Robot]

1 failures in 2627 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform breakdown:

• windows7-32-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1508435&startday=2019-01-07&endday=2019-01-13&tree=all