1532180 - Intermittent devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Test timed out -

Status: RESOLVED FIXED

(DevTools :: Debugger, defect, P1)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: rgurzau [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=231578639&repo=mozilla-central

https://queue.taskcluster.net/v1/task/XZjZu5aJS3GKsO_4QBWPiQ/runs/0/artifacts/public/logs/live_backing.log

23:10:20 INFO - Finished waiting for state change: selected source
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | client is paused -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Got at least one frame -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Frame has correct source -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Frame has correct line -
23:10:20 INFO - Test resuming in the mainThread
23:10:20 INFO - Resuming from 11
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | client is not paused -
23:10:20 INFO - Test pausing in both workers
23:10:20 INFO - Waiting for ADD_BREAKPOINT to dispatch 1 time(s)
23:10:20 INFO - ADD_BREAKPOINT dispatched 1 time(s)
23:10:20 INFO - Invoking in tab: sayHello()
23:10:20 INFO - Waiting for state change: paused
23:10:20 INFO - Finished waiting for state change: paused
23:10:20 INFO - Finished waiting for state change: paused
23:10:20 INFO - Waiting for state change: selected source
23:10:20 INFO - Finished waiting for state change: selected source
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | client is paused -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Got at least one frame -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Frame has correct source -
23:10:20 INFO - TEST-PASS | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Frame has correct line -
23:10:20 INFO - Waiting for state change: paused
23:10:20 INFO - Buffered messages finished
23:10:20 INFO - TEST-UNEXPECTED-FAIL | devtools/client/debugger/new/test/mochitest/browser_dbg-windowless-workers.js | Test timed out -
23:10:20 INFO - GECKO(6372) | console.warn: "Error while detaching the thread front: 'detach' request packet to 'server1.conn5.child1/context22' can't be sent as the connection is closed."
23:10:20 INFO - GECKO(6372) | console.warn: "Error while detaching the thread front: 'detach' request packet to 'server1.conn5.child1/workerTarget25/context1' can't be sent as the connection is closed."
23:10:20 INFO - GECKO(6372) | console.warn: "Error while detaching the thread front: 'detach' request packet to 'server1.conn5.child1/workerTarget26/context1' can't be sent as the connection is closed."
23:10:20 INFO - Removing tab.
23:10:20 INFO - Waiting for event: 'TabClose' on [object XULElement].
23:10:20 INFO - Got event: 'TabClose' on [object XULElement].
23:10:20 INFO - Tab removed and finished closing

Comment 1

[Intermittent Failures Robot]

2 failures in 2981 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 1

Platform breakdown:

• windows10-64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-02-25&endday=2019-03-03&tree=all

Comment 3

[Intermittent Failures Robot]

21 failures in 592 pushes (0.035 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 15
• mozilla-central: 3
• mozilla-inbound: 2
• try: 1

Platform breakdown:

• windows10-64: 9
• linux64: 7
• windows7-32: 1
• linux32: 2
• linux32-nightly: 1
• osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-06&endday=2019-03-06&tree=all

Comment 4

[Intermittent Failures Robot]

70 failures in 506 pushes (0.138 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 59
• mozilla-central: 6
• mozilla-inbound: 5

Platform breakdown:

• linux64: 10
• windows10-64: 23
• osx-10-10: 20
• windows7-32: 16
• windows10-64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-07&endday=2019-03-07&tree=all

Comment 5

[Intermittent Failures Robot]

129 failures in 516 pushes (0.25 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 71
• mozilla-inbound: 31
• mozilla-central: 23
• try: 4

Platform breakdown:

• windows10-64: 55
• osx-10-10: 31
• windows7-32: 16
• linux64: 19
• linux32: 2
• windows10-64-ccov: 1
• linux64-ccov: 1
• linux64-nightly: 1
• macosx64-devedition: 2
• macosx64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-08&endday=2019-03-08&tree=all

Comment 6

[Cristina Coroiu [:ccoroiu]]

Attachment: Bug 1532180 - disable browser_dbg-windowless-workers.js for frequent failures

Comment 7

[Jason Laster [:jlast]]

I believe this will fix it.
https://bugzilla.mozilla.org/show_bug.cgi?id=1533945

Comment 8

[Cristina Coroiu [:ccoroiu]]

Joel,
Please take a look at Comment 7, before reviewing my patch.
Thank you

Comment 9

[Intermittent Failures Robot]

27 failures in 203 pushes (0.133 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 22
• try: 1
• mozilla-inbound: 4

Platform breakdown:

• osx-10-10: 13
• windows7-32: 3
• linux64: 3
• windows10-64: 6
• linux32: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-09&endday=2019-03-09&tree=all

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

bug 1533945 landed March 8th and March 9th there were many failures on autoland still :(

Comment 11

[Jason Laster [:jlast]]

It looks like it is still pretty consistent:

https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-02-21&endday=2019-03-09&tree=trunk&bug=1532180

Comment 12

[Intermittent Failures Robot]

259 failures in 2818 pushes (0.092 failures/push) were associated with this bug in the last 7 days.

This is the #2 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 176
• mozilla-inbound: 43
• mozilla-central: 34
• try: 6

Platform breakdown:

• windows10-64: 98
• linux64: 42
• windows10-64-ccov: 2
• osx-10-10: 66
• linux32: 8
• windows7-32: 36
• linux32-nightly: 1
• windows10-64-nightly: 1
• linux64-ccov: 1
• linux64-nightly: 1
• macosx64-devedition: 2
• macosx64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 13

[Andreea Pavel [:apavel]]

Jason are you working on this or should we land Cristina's disabling patch?

Comment 14

[Jason Laster [:jlast]]

:apavel, we're working on a fix. This looks like a legitimate issue that we want to fix asap.

Comment 15

[Jason Laster [:jlast]]

CCing Brian who is looking into it.

Comment 16

[Andreea Pavel [:apavel]]

Thanks Jason. I keep asking as this has reached the disabling threshold and got the stockwell disable recommended tag, reason why Cristina made the disabling patch. We won't land it.

Comment 17

[Brian Hackett [Laid off!]]

Attachment: Bug 1532180 - Always specify thread when accessing debugger pause state, r=jlast.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f996288d2310
Always specify thread when accessing debugger pause state, r=jlast.

Comment 19

[Cristina Coroiu [:ccoroiu]]

Backed out changeset f996288d2310 (bug 1532180) for damp failures at debugger/custom.js

Backout: https://hg.mozilla.org/integration/mozilla-inbound/rev/00b6a6203f0f108344486f73490c642f0ab11ff2

Failure push: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&selectedJob=233556723&revision=f996288d231058552a104b9a838944a6d0e87016

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=233556723&repo=mozilla-inbound&lineNumber=910

06:51:17 INFO - PID 1765 | Waiting until: has file metadata
06:51:18 INFO - PID 1765 | Finished Waiting until: has file metadata
06:51:18 INFO - PID 1765 | 'custom.jsdebugger.open.settle.DAMP' took 0.26888300001155585ms.
06:51:18 INFO - PID 1765 | Garbage collect
06:51:20 INFO - PID 1765 | Reload page on 'custom.jsdebugger'
06:51:20 INFO - PID 1765 | Waiting for state change: count sources
06:51:21 INFO - PID 1765 | Finished waiting for state change: count sources
06:51:22 INFO - PID 1765 | Waiting until: text is visible
06:51:22 INFO - PID 1765 | Finished Waiting until: text is visible
06:51:22 INFO - PID 1765 | Waiting until: has file metadata
06:51:22 INFO - PID 1765 | Finished Waiting until: has file metadata
06:51:22 INFO - PID 1765 | Wait for pending paints on 'custom.jsdebugger'
06:51:22 INFO - PID 1765 | 'custom.jsdebugger.reload.settle.DAMP' took 0.2820599999977276ms.
06:51:22 INFO - PID 1765 | Waiting for debugger panel
06:51:22 INFO - PID 1765 | Creating context
06:51:22 INFO - PID 1765 | Pausing debugger
06:51:22 INFO - PID 1765 | add breakpoint
06:51:22 INFO - PID 1765 | Selecting source: App.js
06:51:22 INFO - PID 1765 | Waiting for state change: selected source
06:51:22 INFO - PID 1765 | Finished waiting for state change: selected source
06:51:22 INFO - PID 1765 | Run function in content process: window.hitBreakpoint()
06:56:15 INFO - PID 1765 | TEST-UNEXPECTED-FAIL | damp | debugger/custom.js: Test timed out
06:56:15 INFO - Terminating psutil.Process(pid=1765, name='firefox', started='06:49:38')
06:56:15 INFO - PID 1765 | Exiting due to channel error.
06:56:15 INFO - PID 1765 | [GFX1-]: Receive IPC close with reason=AbnormalShutdown
06:56:15 INFO - PID 1765 | Exiting due to channel error.
06:56:15 INFO - PID 1765 | [GFX1-]: Receive IPC close with reason=AbnormalShutdown
06:56:15 INFO - PID 1765 | Exiting due to channel error.
06:56:15 INFO - PID 1765 | Exiting due to channel error.
06:56:15 INFO - TEST-UNEXPECTED-ERROR | damp | unexpected error
06:56:15 ERROR - Traceback (most recent call last):
06:56:15 INFO - File "/Users/cltbld/tasks/task_1552457951/build/tests/talos/talos/run_tests.py", line 300, in run_tests
06:56:15 INFO - talos_results.add(mytest.runTest(browser_config, test))
06:56:15 INFO - File "/Users/cltbld/tasks/task_1552457951/build/tests/talos/talos/ttest.py", line 64, in runTest
06:56:15 INFO - return self._runTest(browser_config, test_config, setup)
06:56:15 INFO - File "/Users/cltbld/tasks/task_1552457951/build/tests/talos/talos/ttest.py", line 215, in _runTest
06:56:15 INFO - debugger_args=browser_config['debugger_args']
06:56:15 INFO - File "/Users/cltbld/tasks/task_1552457951/build/tests/talos/talos/talos_process.py", line 156, in run_browser
06:56:15 INFO - raise TalosError("unexpected error")
06:56:15 INFO - TalosError: unexpected error
06:56:15 INFO - TEST-INFO took 397144ms
06:56:15 INFO - SUITE-END | took 397s
06:56:15 ERROR - Return code: 2
06:56:15 WARNING - setting return code to 2
06:56:15 ERROR - # TBPL FAILURE #

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/12b945b6f707
Always specify thread when accessing debugger pause state, r=jlast.

Comment 21

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/12b945b6f707

Comment 22

[Intermittent Failures Robot]

30 failures in 2771 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 16
• mozilla-central: 9
• mozilla-inbound: 5

Platform breakdown:

• osx-10-10: 5
• linux64: 12
• windows10-64: 10
• windows10-64-ccov: 1
• linux32: 1
• linux64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1532180&startday=2019-03-11&endday=2019-03-17&tree=all