1585114 - Intermittent fission GECKO(5067) | Assertion failure: !IsShuttingDown(), at /builds/worker/workspace/build/src/dom/ipc/ContentChild.cpp:1865

Status: RESOLVED FIXED

(Core :: DOM: Content Processes, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=269087908&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/HJMqsZF4THGsJXXigW2EKA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-09-30T17:05:30.030Z] 17:05:30 INFO - TEST-START | netwerk/cookie/test/browser/browser_sharedWorker.js
[task 2019-09-30T17:05:30.071Z] 17:05:30 INFO - GECKO(5067) | ++DOMWINDOW == 1 (0x7fcb308aff20) [pid = 6693] [serial = 1] [outer = (nil)]
[task 2019-09-30T17:05:30.078Z] 17:05:30 INFO - GECKO(5067) | ++DOMWINDOW == 2 (0x7fcb16e17400) [pid = 6693] [serial = 2] [outer = 0x7fcb308aff20]
[task 2019-09-30T17:05:30.160Z] 17:05:30 INFO - GECKO(5067) | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /tmp/tmpBSrDJu.mozrunner/runtests_leaks_tab_pid6901.log
[task 2019-09-30T17:05:30.221Z] 17:05:30 INFO - GECKO(5067) | --DOCSHELL 0x7fe2e9cb3800 == 0 [pid = 6875] [id = {fb442773-4503-4bd7-8365-c47b81afd5b3}] [url = http://example.com/browser/netwerk/cookie/test/browser/file_empty.html]
[task 2019-09-30T17:05:30.317Z] 17:05:30 INFO - GECKO(5067) | Couldn't convert chrome URL: chrome://branding/locale/brand.properties
[task 2019-09-30T17:05:30.353Z] 17:05:30 INFO - GECKO(5067) | --DOMWINDOW == 2 (0x7fe2e9ce9d40) [pid = 6875] [serial = 1] [outer = (nil)] [url = http://example.com/browser/netwerk/cookie/test/browser/file_empty.html]
[task 2019-09-30T17:05:30.353Z] 17:05:30 INFO - GECKO(5067) | --DOMWINDOW == 1 (0x7fe2e9c57400) [pid = 6875] [serial = 3] [outer = (nil)] [url = http://example.com/browser/netwerk/cookie/test/browser/file_empty.html]
[task 2019-09-30T17:05:30.353Z] 17:05:30 INFO - GECKO(5067) | --DOMWINDOW == 0 (0x7fe2e9b05000) [pid = 6875] [serial = 2] [outer = (nil)] [url = about:blank]

[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | JavaScript error: resource://gre/modules/ProcessSelector.jsm, line 56: TypeError: process is null
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | [Parent 5067, Main Thread] WARNING: nsIContentProcessProvider failed to return a process: file /builds/worker/workspace/build/src/dom/ipc/ContentParent.cpp, line 897
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | Assertion failure: !IsShuttingDown(), at /builds/worker/workspace/build/src/dom/ipc/ContentChild.cpp:1865
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #01: mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:fd8cde07bc12da00eb110c7ee29edc1f156c76a3c38880b5eb998e6f5a508d631a6ba97e85c721400241c86c5644225bfec5fe45de8b84ef4fe326e48c5521d0/ipc/ipdl/PContentChild.cpp::7953]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #02: mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) [ipc/glue/MessageChannel.cpp:2186]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #03: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) [ipc/glue/MessageChannel.cpp:2112]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #04: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) [ipc/glue/MessageChannel.cpp:0]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #05: mozilla::ipc::MessageChannel::MessageTask::Run() [ipc/glue/MessageChannel.cpp:1987]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #06: mozilla::SchedulerGroup::Runnable::Run() [xpcom/threads/SchedulerGroup.cpp:295]
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.793Z] 17:05:33 INFO - GECKO(5067) | #07: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1225]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #08: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:486]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #09: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:0]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #10: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:315]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #11: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:291]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #12: nsBaseAppShell::Run() [widget/nsBaseAppShell.cpp:139]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #13: XRE_RunAppShell() [toolkit/xre/nsEmbedFunctions.cpp:934]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #14: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:238]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO - GECKO(5067) | #15: MessageLoop::RunInternal() [ipc/chromium/src/base/message_loop.cc:315]
[task 2019-09-30T17:05:33.794Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.801Z] 17:05:33 INFO - GECKO(5067) | #16: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:291]
[task 2019-09-30T17:05:33.801Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.801Z] 17:05:33 INFO - GECKO(5067) | #17: XRE_InitChildProcess(int, char**, XREChildData const*) [toolkit/xre/nsEmbedFunctions.cpp:769]
[task 2019-09-30T17:05:33.801Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.943Z] 17:05:33 INFO - GECKO(5067) | #18: content_process_main(mozilla::Bootstrap*, int, char**) [ipc/contentproc/plugin-container.cpp:57]
[task 2019-09-30T17:05:33.943Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.943Z] 17:05:33 INFO - GECKO(5067) | #19: main [browser/app/nsBrowserApp.cpp:272]
[task 2019-09-30T17:05:33.943Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.944Z] 17:05:33 INFO - GECKO(5067) | #20: libc.so.6 + 0x20830
[task 2019-09-30T17:05:33.945Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.945Z] 17:05:33 INFO - GECKO(5067) | #21: _start
[task 2019-09-30T17:05:33.945Z] 17:05:33 INFO -
[task 2019-09-30T17:05:33.947Z] 17:05:33 INFO - GECKO(5067) | ###!!! [Parent][MessageChannel] Error: (msgtype=0x1E0073,name=PBrowser::Msg_UpdateDimensions) Channel error: cannot send/recv
[task 2019-09-30T17:05:33.948Z] 17:05:33 INFO - GECKO(5067) | ###!!! [Parent][RunMessage] Error: Channel error: cannot send/recv

[task 2019-09-30T17:07:00.509Z] 17:07:00 INFO - TEST-PASS | netwerk/cookie/test/browser/browser_sharedWorker.js | SharedWorker is allowed - true == true -
[task 2019-09-30T17:07:00.509Z] 17:07:00 INFO - Leaving test bound
[task 2019-09-30T17:07:00.515Z] 17:07:00 INFO - Entering test bound
[task 2019-09-30T17:07:00.518Z] 17:07:00 INFO - Starting SharedWorker: ({fromBehavior:2, toBehavior:2, fromPermission:1, toPermission:0})
[task 2019-09-30T17:07:00.519Z] 17:07:00 INFO - Console message: [JavaScript Error: "Loading Worker from “https://example.com/browser/netwerk/cookie/test/browser/a.js” was blocked because of a disallowed MIME type (“text/html”)."]
[task 2019-09-30T17:07:00.519Z] 17:07:00 INFO - Console message: [JavaScript Error: "TypeError: process is null" {file: "resource://gre/modules/ProcessSelector.jsm" line: 56}]
[task 2019-09-30T17:07:00.519Z] 17:07:00 INFO - Buffered messages logged at 17:05:34
[task 2019-09-30T17:07:00.520Z] 17:07:00 INFO - Console message: [JavaScript Error: "remote browser crashed while on about:blank
[task 2019-09-30T17:07:00.521Z] 17:07:00 INFO - " {file: "chrome://mochikit/content/mochitest-e10s-utils.js" line: 10}]
[task 2019-09-30T17:07:00.522Z] 17:07:00 INFO - e10s_init/<@chrome://mochikit/content/mochitest-e10s-utils.js:10:10
[task 2019-09-30T17:07:00.522Z] 17:07:00 INFO - EventListener.handleEvente10s_init@chrome://mochikit/content/mochitest-e10s-utils.js:6:10
[task 2019-09-30T17:07:00.523Z] 17:07:00 INFO - testInit@chrome://mochikit/content/browser-test.js:131:5
[task 2019-09-30T17:07:00.524Z] 17:07:00 INFO - setTimeout handler@chrome://mochikit/content/browser-test.js:41:11
[task 2019-09-30T17:07:00.524Z] 17:07:00 INFO - loadChromeScripts@jar:file:///tmp/tmpBSrDJu.mozrunner/extensions/[e-mail]!/api.js:21:25
[task 2019-09-30T17:07:00.525Z] 17:07:00 INFO - loadMochitest@jar:file:///tmp/tmpBSrDJu.mozrunner/extensions/[e-mail]!/api.js:143:20
[task 2019-09-30T17:07:00.525Z] 17:07:00 INFO - @tests/mochitest/runtests.py:1810:5
[task 2019-09-30T17:07:00.526Z] 17:07:00 INFO - @tests/mochitest/runtests.py:1811:8
[task 2019-09-30T17:07:00.526Z] 17:07:00 INFO - evaluate.sandbox/promise<@chrome://marionette/content/evaluate.js:138:10
[task 2019-09-30T17:07:00.526Z] 17:07:00 INFO - evaluate.sandbox@chrome://marionette/content/evaluate.js:114:17
[task 2019-09-30T17:07:00.527Z] 17:07:00 INFO - GeckoDriver.prototype.execute_@chrome://marionette/content/driver.js:1070:28
[task 2019-09-30T17:07:00.527Z] 17:07:00 INFO - async*GeckoDriver.prototype.executeScript@chrome://marionette/content/driver.js:945:30
[task 2019-09-30T17:07:00.528Z] 17:07:00 INFO - despatch@chrome://marionette/content/server.js:305:40
[task 2019-09-30T17:07:00.528Z] 17:07:00 INFO - execute@chrome://marionette/content/server.js:275:16
[task 2019-09-30T17:07:00.528Z] 17:07:00 INFO - onPacket/<@chrome://marionette/content/server.js:248:20
[task 2019-09-30T17:07:00.529Z] 17:07:00 INFO - onPacket@chrome://marionette/content/server.js:249:9
[task 2019-09-30T17:07:00.529Z] 17:07:00 INFO - _onJSONObjectReady/<@chrome://marionette/content/transport.js:501:20
[task 2019-09-30T17:07:00.529Z] 17:07:00 INFO -
[task 2019-09-30T17:07:00.530Z] 17:07:00 INFO - Buffered messages finished
[task 2019-09-30T17:07:00.531Z] 17:07:00 INFO - TEST-UNEXPECTED-FAIL | netwerk/cookie/test/browser/browser_sharedWorker.js | Test timed out -
[task 2019-09-30T17:07:00.532Z] 17:07:00 INFO - GECKO(5067) | MEMORY STAT | vsize 2979MB | residentFast 326MB | heapAllocated 100MB
[task 2019-09-30T17:07:00.533Z] 17:07:00 INFO - TEST-OK | netwerk/cookie/test/browser/browser_sharedWorker.js | took 90022ms
[task 2019-09-30T17:07:00.534Z] 17:07:00 INFO - Not taking screenshot here: see the one that was previously logged
[task 2019-09-30T17:07:00.534Z] 17:07:00 INFO - TEST-UNEXPECTED-FAIL | netwerk/cookie/test/browser/browser_sharedWorker.js | Found a tab after previous test timed out: about:blank -
[task 2019-09-30T17:07:00.535Z] 17:07:00 INFO - GECKO(5067) | ++DOCSHELL 0x7f0cb83b2800 == 1 [pid = 6984] [id = {534e9722-6113-47fb-8571-68edc15d0c76}]
[task 2019-09-30T17:07:00.536Z] 17:07:00 INFO - checking window state
[task 2019-09-30T17:07:00.536Z] 17:07:00 INFO - GECKO(5067) | ++DOMWINDOW == 1 (0x7f0cb821ff20) [pid = 6984] [serial = 1] [outer = (nil)]
[task 2019-09-30T17:07:00.537Z] 17:07:00 INFO - GECKO(5067) | ++DOMWINDOW == 2 (0x7f0cb8332c00) [pid = 6984] [serial = 2] [outer = 0x7f0cb821ff20]

Comment 1

[Intermittent Failures Robot]

12 failures in 4292 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• autoland: 8
• mozilla-inbound: 1

Platform breakdown:

• linux64: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-09-30&endday=2019-10-06&tree=all

Comment 2

[Intermittent Failures Robot]

14 failures in 3801 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 14

Platform breakdown:

• linux64: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-10-07&endday=2019-10-13&tree=all

Comment 3

[Intermittent Failures Robot]

6 failures in 3964 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6

Platform breakdown:

• linux64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-10-14&endday=2019-10-20&tree=all

Comment 4

[Intermittent Failures Robot]

2 failures in 4182 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-10-21&endday=2019-10-27&tree=all

Comment 5

[Intermittent Failures Robot]

18 failures in 783 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 18

Platform breakdown:

• linux64: 18

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-01&endday=2019-11-01&tree=all

Comment 6

[Intermittent Failures Robot]

38 failures in 4314 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 36
• mozilla-central: 2

Platform breakdown:

• linux64: 38

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-10-28&endday=2019-11-03&tree=all

Comment 7

[Intermittent Failures Robot]

26 failures in 1017 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 23
• mozilla-inbound: 1
• mozilla-central: 2

Platform breakdown:

• linux64: 26

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-05&endday=2019-11-05&tree=all

Comment 8

[Intermittent Failures Robot]

24 failures in 895 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 23
• mozilla-central: 1

Platform breakdown:

• linux64: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-08&endday=2019-11-08&tree=all

Comment 9

[Intermittent Failures Robot]

80 failures in 4850 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #20 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 68
• mozilla-inbound: 3
• mozilla-central: 9

Platform breakdown:

• linux64: 79
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-04&endday=2019-11-10&tree=all

Comment 10

[Andreea Pavel [:apavel]]

There are 92 total failures in the last 7 days and 144 total failures in the last 30.

All failures are on fission and happen on linux64 debug,

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=275829060&repo=autoland&lineNumber=22132

[task 2019-11-12T16:57:10.607Z] 16:57:10 INFO - TEST-START | netwerk/cookie/test/browser/browser_sharedWorker.js
[task 2019-11-12T16:57:10.684Z] 16:57:10 INFO - GECKO(7973) | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to /tmp/tmpU9ctbs.mozrunner/runtests_leaks_tab_pid9965.log
[task 2019-11-12T16:57:10.700Z] 16:57:10 INFO - GECKO(7973) | [Child 9965, Main Thread] WARNING: No CID found when attempting to map contract ID: file /builds/worker/workspace/build/src/xpcom/components/nsComponentManager.cpp, line 729
[task 2019-11-12T16:57:10.735Z] 16:57:10 INFO - GECKO(7973) | Couldn't convert chrome URL: chrome://branding/locale/brand.properties
[task 2019-11-12T16:57:10.897Z] 16:57:10 INFO - GECKO(7973) | Couldn't convert chrome URL: chrome://branding/locale/brand.properties
[task 2019-11-12T16:57:10.920Z] 16:57:10 INFO - GECKO(7973) | [Child 9954, Main Thread] WARNING: could not set real-time limit at process startup: file /builds/worker/workspace/build/src/dom/ipc/ContentChild.cpp, line 1799
[task 2019-11-12T16:57:10.936Z] 16:57:10 INFO - GECKO(7973) | ++DOCSHELL 0x7fade140b800 == 1 [pid = 9954] [id = {684107e1-6f4f-4e86-98f6-6205d132e9f4}]

[task 2019-11-12T16:57:14.243Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, IPDL Background] WARNING: The service must exist if we want to close an existing MessagePort.: file /builds/worker/workspace/build/src/dom/messagechannel/MessagePortParent.cpp, line 154
[task 2019-11-12T16:57:14.283Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, Main Thread] WARNING: Suboptimal indexes for the SQL statement 0x7f5efd8b01b0 (http://mzl.la/1FuID0j).: file /builds/worker/workspace/build/src/storage/mozStoragePrivateHelpers.cpp, line 108
[task 2019-11-12T16:57:14.311Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x804B001D: file /builds/worker/workspace/build/src/netwerk/protocol/http/nsHttpChannel.cpp, line 1792
[task 2019-11-12T16:57:14.419Z] 16:57:14 INFO - GECKO(7973) | JavaScript error: resource://gre/modules/ProcessSelector.jsm, line 56: TypeError: process is null
[task 2019-11-12T16:57:14.419Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, Main Thread] WARNING: nsIContentProcessProvider failed to return a process: file /builds/worker/workspace/build/src/dom/ipc/ContentParent.cpp, line 901
[task 2019-11-12T16:57:14.422Z] 16:57:14 INFO - GECKO(7973) | Assertion failure: !IsShuttingDown(), at /builds/worker/workspace/build/src/dom/ipc/ContentChild.cpp:1876
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #01: mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) [s3:gecko-generated-sources:59f60bf949d553fd8887740bf136e8047d39982ec7dea37940b4ba2b8c784de28c35dce97966d00fd4411205ed7dae3e85fddda4a5b75bc11f78f507474d7cc0/ipc/ipdl/PContentChild.cpp::8220]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #02: mozilla::ipc::MessageChannel::DispatchAsyncMessage(mozilla::ipc::ActorLifecycleProxy*, IPC::Message const&) [ipc/glue/MessageChannel.cpp:2209]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #03: mozilla::ipc::MessageChannel::DispatchMessage(IPC::Message&&) [ipc/glue/MessageChannel.cpp:2133]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #04: mozilla::ipc::MessageChannel::RunMessage(mozilla::ipc::MessageChannel::MessageTask&) [ipc/glue/MessageChannel.cpp:0]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #05: mozilla::ipc::MessageChannel::MessageTask::Run() [ipc/glue/MessageChannel.cpp:2005]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #06: mozilla::SchedulerGroup::Runnable::Run() [xpcom/threads/SchedulerGroup.cpp:295]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #07: nsThread::ProcessNextEvent(bool, bool*) [xpcom/threads/nsThread.cpp:1250]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #08: NS_ProcessNextEvent(nsIThread*, bool) [xpcom/threads/nsThreadUtils.cpp:486]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO - GECKO(7973) | #09: mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:88]
[task 2019-11-12T16:57:14.427Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.428Z] 16:57:14 INFO - GECKO(7973) | #10: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:291]
[task 2019-11-12T16:57:14.428Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.429Z] 16:57:14 INFO - GECKO(7973) | #11: nsBaseAppShell::Run() [widget/nsBaseAppShell.cpp:139]
[task 2019-11-12T16:57:14.429Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.430Z] 16:57:14 INFO - GECKO(7973) | #12: XRE_RunAppShell() [toolkit/xre/nsEmbedFunctions.cpp:934]
[task 2019-11-12T16:57:14.431Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.433Z] 16:57:14 INFO - GECKO(7973) | #13: mozilla::ipc::MessagePumpForChildProcess::Run(base::MessagePump::Delegate*) [ipc/glue/MessagePump.cpp:238]
[task 2019-11-12T16:57:14.434Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.438Z] 16:57:14 INFO - GECKO(7973) | #14: MessageLoop::Run() [ipc/chromium/src/base/message_loop.cc:291]
[task 2019-11-12T16:57:14.439Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.439Z] 16:57:14 INFO - GECKO(7973) | #15: XRE_InitChildProcess(int, char**, XREChildData const*) [toolkit/xre/nsEmbedFunctions.cpp:769]
[task 2019-11-12T16:57:14.440Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.440Z] 16:57:14 INFO - GECKO(7973) | #16: content_process_main(mozilla::Bootstrap*, int, char**) [ipc/contentproc/plugin-container.cpp:57]
[task 2019-11-12T16:57:14.441Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.441Z] 16:57:14 INFO - GECKO(7973) | #17: main [browser/app/nsBrowserApp.cpp:272]
[task 2019-11-12T16:57:14.441Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.442Z] 16:57:14 INFO - GECKO(7973) | #18: libc.so.6 + 0x20830
[task 2019-11-12T16:57:14.442Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.442Z] 16:57:14 INFO - GECKO(7973) | #19: _start
[task 2019-11-12T16:57:14.442Z] 16:57:14 INFO -
[task 2019-11-12T16:57:14.507Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/workspace/build/src/ipc/glue/ProtocolUtils.cpp, line 481
[task 2019-11-12T16:57:14.508Z] 16:57:14 INFO - GECKO(7973) | [Parent 7973, Main Thread] WARNING: IPC message discarded: actor cannot send: file /builds/worker/workspace/build/src/ipc/glue/ProtocolUtils.cpp, line 481

Jim can you please assign someone to take a look?

Comment 11

[Neha Kochar [:neha]]

Andrew, can you look into this since it looks to be IPC related?

Comment 12

[Andrew McCreight [:mccr8]]

Well, if I knew how to fix this, I would have fixed it instead of needinfoing you. :) I'll see if I can figure out what this code is. The assertion was added in bug 1331685. Bug 1331685 comment 8 says "We should just assert !IsShuttingDown here", but there's no explanation. I think we crash if a constructor fails, so returning instead of asserting likely won't help anything. Bug 1576714 landed around when this showed up, but the periods where it was backed out don't seem to correspond to when this error was seen.

Comment 13

[Andrew McCreight [:mccr8]]

I was able to reproduce this locally with
./mach mochitest --verify --enable-fission --headless netwerk/cookie/test/browser/browser_sharedWorker.js

Comment 14

[Matt Woodrow (:mattwoodrow)]

It looks like we're crashing because we ran PContentParent::SendConstructBrowser after PContentParent::SendShutdown.

Given the error message "[Parent 7973, Main Thread] WARNING: nsIContentProcessProvider failed to return a process: file /builds/worker/workspace/build/src/dom/ipc/ContentParent.cpp, line 901", it seems that we must have obtained our ContentParent here - https://searchfox.org/mozilla-central/rev/6566d92dd46417a2f57e75c515135ebe84c9cef5/dom/ipc/ContentParent.cpp#900

It looks like the cpp->ProvideProcess path (which we're not taking in this instance) checks for shut down parents, since ContentParent::ShutDownProcess call ProcessDied on the scriptable helper (and then clears it).

I can't see anything stopping us from picking a ContentParent that is mid-shutdown from the random selection path, since it appears that we don't remove them from the pool until ActorDestroy (which is when we finish shutdown, not start it).

I think we should be checking for !p->mShutdownPending in ContentParent::MinTabSelect.

Comment 15

[Andrew McCreight [:mccr8]]

Yeah, so the actual error in the JSM is a line earlier and is this:
JavaScript error: resource://gre/modules/ProcessSelector.jsm, line 56: TypeError: process is null
At least part of the problem is that we're not null checking cp->mScriptableHelper when we push onto infos. Fixing that at least let me pass a verify run. But yeah, it makes sense that the actual underlying problem is that this fallback case is broken.

Comment 16

[Andrew McCreight [:mccr8]]

Attachment: Bug 1585114 - Don't return shutdown pending processes from ContentParent::GetNewOrUsedBrowserProcess.

ContentParent::GetNewOrUsedBrowserProcess failed to handle content
parents that were in the middle of being shut down in two ways.

First, it would end up trying to pass them in to the provide process
method, which resulted in null entries the infos array, which caused
the JS implementation to throw an error.

Second, when the nsIContentProcessProvider implementation threw an
error, we'd fall back to MinTabSelect(), but that didn't exclude
content parents that were being shutdown.

Comment 17

[Andrew McCreight [:mccr8]]

Thanks for the help, Matt. My patch contains fixes for both issues. Either one of them fixes the assertion. I looked through GetNewOrUsedBrowserProcess() and it looks like the other cases properly deal with content parents where shutdown is pending.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/37b70cca0d2f
Don't return shutdown pending processes from ContentParent::GetNewOrUsedBrowserProcess. r=mattwoodrow

Comment 19

[Intermittent Failures Robot]

22 failures in 746 pushes (0.029 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 22

Platform breakdown:

• linux64: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-12&endday=2019-11-12&tree=all

Comment 20

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/37b70cca0d2f

Comment 21

[Chris Peterson [:cpeterson]]

Retroactively moving fixed bugs whose summaries mention "Fission" (or other Fission-related keywords) but are not assigned to a Fission Milestone to an appropriate Fission Milestone.

This will generate a lot of bugmail, so you can filter your bugmail for the following UUID and delete them en masse:

0ee3c76a-bc79-4eb2-8d12-05dc0b68e732

Comment 22

[Intermittent Failures Robot]

31 failures in 4207 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 30
• mozilla-central: 1

Platform breakdown:

• linux64: 30
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-11&endday=2019-11-17&tree=all

Comment 23

[Andrew McCreight [:mccr8]]

This is a bug with non-Fission e10s, too, but we just don't create as many processes, so it doesn't show up as much. I don't know if it is worth backporting or not. Probably not?

Comment 24

[Andrew McCreight [:mccr8]]

(Probably not, given that this bug has been present for the entirety of the time we've been shipping e10s and nobody has noticed it before.)

Comment 25

[Intermittent Failures Robot]

1 failures in 5444 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-11-18&endday=2019-11-24&tree=all

Comment 26

[Intermittent Failures Robot]

1 failures in 5144 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-12-02&endday=2019-12-08&tree=all

Comment 27

[Intermittent Failures Robot]

1 failures in 4867 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-12-09&endday=2019-12-15&tree=all

Comment 28

[Intermittent Failures Robot]

1 failures in 1365 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2019-12-23&endday=2019-12-29&tree=all

Comment 29

[Intermittent Failures Robot]

3 failures in 4472 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• linux64-qr: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 30

[Intermittent Failures Robot]

1 failures in 3773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1585114&startday=2020-01-13&endday=2020-01-19&tree=all