1429361 - Intermittent devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_network_messages_expand.js | Test timed out -

Status: RESOLVED FIXED

(DevTools :: Console, defect, P5)

Description

[Natalia Csoregi [:nataliaCs]]

+++ This bug was initially created as a clone of Bug #1398904 +++

Filed by: wkocher [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=130125086&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/O-O4QJRXTTyRCL5pp2NFzA/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Intermittent Failures Robot]

16 failures in 149 pushes (0.107 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 7
* autoland: 7
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 7
* linux64: 4
* linux32: 2
* windows10-64: 1
* macosx64-devedition: 1
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-11&endday=2018-01-11&tree=all

Comment 2

[Intermittent Failures Robot]

51 failures in 788 pushes (0.065 failures/push) were associated with this bug in the last 7 days. 

This is the #24 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 23
* mozilla-inbound: 13
* mozilla-central: 10
* try: 4
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 18
* linux64: 13
* linux64-stylo-disabled: 5
* osx-10-10: 4
* linux32: 4
* windows10-64: 2
* windows7-32-nightly: 1
* macosx64-devedition: 1
* linux64-nightly: 1
* linux64-devedition: 1
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-08&endday=2018-01-14&tree=all

Comment 3

[Tiberius Oros[:tiberius_oros]]

There have been 51 failures in the last week.

Failures per platform:

Linux x64: 20
Windows 7: 13
OS X 10.10: 6
Linux: 5
linux64-stylo-disabled: 4
windows10-64: 2
linux64-ccov: 1

Failures per build type:

debug: 27
opt: 17
pgo: 7

I see this had an increase in failures on January 14th. Since January the 15th it dropped to an average of 4-5 failures / day.

Here is a recent log file and a snippet with the failure:

https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=156986412&lineNumber=16829

[task 2018-01-17T23:15:30.531Z] 23:15:30     INFO - Buffered messages logged at 23:13:59
16827
[task 2018-01-17T23:15:30.531Z] 23:15:30     INFO - Longer timeout required, waiting longer...  Remaining timeouts: 1
16828
[task 2018-01-17T23:15:30.532Z] 23:15:30     INFO - Buffered messages finished
16829
[task 2018-01-17T23:15:30.532Z] 23:15:30     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_network_messages_expand.js | Test timed out - 
16830
[task 2018-01-17T23:15:30.538Z] 23:15:30     INFO - GECKO(2299) | ++DOMWINDOW == 14 (0x7f9a98de9c00) [pid = 2299] [serial = 608] [outer = 0x7f9ab84c3090]
16831
[task 2018-01-17T23:15:30.539Z] 23:15:30     INFO - GECKO(2299) | --DOCSHELL 0x7f9ab2caa000 == 5 [pid = 2299] [id = {54a13ff4-5790-456d-880d-2338de6d6c9c}]
16832
[task 2018-01-17T23:15:30.540Z] 23:15:30     INFO - Removing tab.

:bgrins,would you please have a look?

Comment 4

[Intermittent Failures Robot]

45 failures in 657 pushes (0.068 failures/push) were associated with this bug in the last 7 days. 

This is the #15 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 16
* mozilla-inbound: 14
* mozilla-central: 9
* try: 4
* mozilla-beta: 2

Platform breakdown:
* linux64: 18
* windows7-32: 6
* linux64-stylo-disabled: 5
* osx-10-10: 4
* linux32: 4
* windows10-64: 2
* windows10-64-ccov: 1
* linux64-devedition: 1
* linux64-ccov: 1
* linux32-nightly: 1
* linux32-devedition: 1
* gecko-decision: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-15&endday=2018-01-21&tree=all

Comment 5

[Cristina Coroiu [:ccoroiu]]

Since 19th January there are 46 failures.
For the failure pattern see Comment 3

:bgrins could you please take a look?

Comment 6

[Intermittent Failures Robot]

40 failures in 701 pushes (0.057 failures/push) were associated with this bug in the last 7 days. 

This is the #27 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 21
* mozilla-inbound: 8
* try: 5
* mozilla-central: 5
* mozilla-beta: 1

Platform breakdown:
* linux64-stylo-disabled: 13
* linux64: 9
* windows7-32: 5
* osx-10-10: 4
* linux32: 3
* windows10-64: 2
* linux32-nightly: 2
* windows7-32-devedition: 1
* windows10-64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-22&endday=2018-01-28&tree=all

Comment 7

[Intermittent Failures Robot]

7 failures in 164 pushes (0.043 failures/push) were associated with this bug yesterday.    

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* mozilla-inbound: 3
* mozilla-central: 2
* autoland: 2

Platform breakdown:
* osx-10-10: 2
* linux64: 2
* windows10-64: 1
* linux64-ccov: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-31&endday=2018-01-31&tree=all

Comment 8

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days there are 37 failures on this bug. Most of the failures happen on linux x32, linux x64. macosx x64, windows 10-64 and windows 7-32
Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=160243698&lineNumber=11966

Here is a snippet from the most recent log: 

11:24:44     INFO -  Buffered messages logged at 11:23:14
11:24:44     INFO -  697 INFO Longer timeout required, waiting longer...  Remaining timeouts: 1
11:24:44     INFO -  Buffered messages finished
11:24:44    ERROR -  698 INFO TEST-UNEXPECTED-FAIL | devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_network_messages_expand.js | Test timed out -
11:24:44     INFO -  GECKO(6448) | [Child 6620, Main Thread] WARNING: NS_ENSURE_TRUE(node) failed: file z:/build/build/src/layout/base/nsDocumentViewer.cpp, line 3713
11:24:44     INFO -  GECKO(6448) | ++DOMWINDOW == 16 (00000158001AEC00) [pid = 6448] [serial = 700] [outer = 0000015807753280]
11:24:44     INFO -  699 INFO Removing tab.
11:24:44     INFO -  700 INFO Waiting for event: 'TabClose' on [object XULElement].
11:24:44     INFO -  701 INFO Got event: 'TabClose' on [object XULElement].

Comment 9

[Intermittent Failures Robot]

48 failures in 735 pushes (0.065 failures/push) were associated with this bug in the last 7 days. 

This is the #14 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 17
* mozilla-central: 11
* mozilla-inbound: 10
* try: 5
* mozilla-beta: 5

Platform breakdown:
* linux64: 15
* linux64-stylo-disabled: 8
* linux32: 7
* windows10-64: 6
* osx-10-10: 5
* windows7-32: 2
* windows7-32-nightly: 1
* windows10-64-ccov: 1
* linux64-devedition: 1
* linux64-ccov: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-01-29&endday=2018-02-04&tree=all

Comment 10

[Cosmin Sabou [:CosminS]]

Attachment: disable_bug1429361.patch

Hello! I've modified the .ini file so that tests would skip on Linux x64 and Win10-64 debug based on this pattern of failures:
 
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-02-04&endday=2018-02-11&tree=all

Could you please take a look and let me know if I've done it correctly? 
Thank you.

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8950057 [details] [diff] [review]
disable_bug1429361.patch

Review of attachment 8950057 [details] [diff] [review]:
-----------------------------------------------------------------

really close, fix the issue and we can get this landed.

::: devtools/client/webconsole/new-console-output/test/mochitest/browser.ini
@@ +309,4 @@
>  [browser_webconsole_network_attach.js]
>  [browser_webconsole_network_exceptions.js]
>  [browser_webconsole_network_messages_expand.js]
> +skip-if = ((os == 'linux') || (os == 'win' && os_version == '10.0' && debug)) && bits == 64 # Bug 1429361, disabled on Linux/Win for frequent failures

I would say linux32 fails often enough, lets only add the bits for windows.

Comment 12

[Intermittent Failures Robot]

35 failures in 702 pushes (0.05 failures/push) were associated with this bug in the last 7 days. 

This is the #37 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-central: 12
* autoland: 10
* mozilla-inbound: 7
* try: 4
* mozilla-beta: 2

Platform breakdown:
* linux64: 12
* linux64-stylo-disabled: 9
* windows10-64: 5
* linux64-ccov: 3
* windows7-32: 1
* osx-10-10: 1
* linux64-nightly: 1
* linux64-devedition: 1
* linux32-nightly: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-02-05&endday=2018-02-11&tree=all

Comment 13

[Cosmin Sabou [:CosminS]]

Attachment: disable_bug1429361.patch

Second try.  I left only the 64 bits for windows. 

+skip-if = (os == 'linux') || (os == 'win' && os_version == '10.0' && debug && bits == 64)  # Bug 1429361, disabled on Linux/Win for frequent failures

Comment 14

[Cosmin Sabou [:CosminS]]

:jmaher, I don't know how to delete or make unavailable the first version of the patch. How do we do that? Is it automatically applied the second one? Thank you.

Comment 15

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8950057 [details] [diff] [review]
disable_bug1429361.patch

there is a trick-

on the attachment, edit 'details', then at the top there is yet another link for 'details', there you can edit the details of the file and make it obsolete.

when uploading a new patch you can select another attachment to be obsolete.

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

landed on mozilla-central:
https://hg.mozilla.org/integration/mozilla-inbound/rev/93227b85dd4425e59eee6958c0db9669dfb01414

Comment 17

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/93227b85dd44

Comment 18

[Intermittent Failures Robot]

30 failures in 685 pushes (0.044 failures/push) were associated with this bug in the last 7 days. 

This is the #50 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 15
* mozilla-inbound: 6
* mozilla-beta: 4
* mozilla-central: 3
* try: 2

Platform breakdown:
* osx-10-10: 11
* linux64-stylo-disabled: 6
* linux64: 6
* windows10-64: 2
* windows7-32: 1
* macosx64-nightly: 1
* linux64-nightly: 1
* linux32-nightly: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1429361&startday=2018-02-12&endday=2018-02-18&tree=all

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3e0c6008d60d