1533651 - Intermittent Assertion failure: false (PSessionStorageObserverChild::SendDeleteMe()), at /builds/worker/workspace/build/src/dom/storage/StorageIPC.cpp:429

Status: RESOLVED FIXED

(Core :: Storage: localStorage & sessionStorage, defect, P2)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: shindli [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=232595626&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/Q1PNTOVDQtuKy4-t6iJ4sg/runs/0/artifacts/public/logs/live_backing.log

05:24:19 INFO - TEST-OK | browser/base/content/test/tabs/browser_new_file_whitelisted_http_tab.js | took 2196ms
05:24:19 INFO - GECKO(894) | ++DOCSHELL 0x11e4a4000 == 5 [pid = 896] [id = {9bd9b22b-4d7c-d74d-a4dd-e7959a30be88}]
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 12 (0x127f28b60) [pid = 896] [serial = 138] [outer = 0x0]
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 13 (0x12970f800) [pid = 896] [serial = 139] [outer = 0x127f28b60]
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 14 (0x129709800) [pid = 896] [serial = 140] [outer = 0x127f28b60]
05:24:19 INFO - checking window state
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 2 (0x10a5e6400) [pid = 904] [serial = 28] [outer = 0x0] [url = about:blank]
05:24:19 INFO - GECKO(894) | [Child 918, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:19 INFO - GECKO(894) | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
05:24:19 INFO - TEST-START | browser/base/content/test/tabs/browser_new_tab_in_privileged_process_pref.js
05:24:19 INFO - GECKO(894) | ++DOCSHELL 0x11bed2800 == 2 [pid = 898] [id = {7b9f8e75-fdbd-8c47-ba1d-74e29f160dcd}]
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 3 (0x115a35200) [pid = 898] [serial = 25] [outer = 0x0]
05:24:19 INFO - GECKO(894) | --DOCSHELL 0x114cc3800 == 0 [pid = 918] [id = {c54d8138-46fe-a541-81f4-008e1f8e02bd}] [url = http://example.org/]
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 2 (0x10e83d200) [pid = 918] [serial = 1] [outer = 0x0] [url = http://example.org/]
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 1 (0x114c8f800) [pid = 918] [serial = 2] [outer = 0x0] [url = about:blank]
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 0 (0x11e4acc00) [pid = 918] [serial = 3] [outer = 0x0] [url = http://example.org/]
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 4 (0x12569f000) [pid = 898] [serial = 26] [outer = 0x115a35200]
05:24:19 INFO - GECKO(894) | nsStringStats
05:24:19 INFO - GECKO(894) | => mAllocCount: 8489
05:24:19 INFO - GECKO(894) | => mReallocCount: 0
05:24:19 INFO - GECKO(894) | => mFreeCount: 8489
05:24:19 INFO - GECKO(894) | => mShareCount: 7481
05:24:19 INFO - GECKO(894) | => mAdoptCount: 608
05:24:19 INFO - GECKO(894) | => mAdoptFreeCount: 612
05:24:19 INFO - GECKO(894) | => Process ID: 918, Thread ID: 140735090127616
05:24:19 INFO - GECKO(894) | ++DOMWINDOW == 5 (0x1256a5c00) [pid = 898] [serial = 27] [outer = 0x115a35200]
05:24:19 INFO - GECKO(894) | [Child 897, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:19 INFO - GECKO(894) | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
05:24:19 INFO - GECKO(894) | [Child 899, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 13 (0x12e949000) [pid = 897] [serial = 117] [outer = 0x0] [url = about:blank]
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 12 (0x126fd7000) [pid = 897] [serial = 122] [outer = 0x0] [url = about:blank]
05:24:19 INFO - GECKO(894) | --DOMWINDOW == 11 (0x115fdf800) [pid = 897] [serial = 119] [outer = 0x0] [url = about:blank]
05:24:19 INFO - GECKO(894) | [Child 900, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:19 INFO - GECKO(894) | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
05:24:20 INFO - GECKO(894) | --DOMWINDOW == 11 (0x11ecb2400) [pid = 899] [serial = 143] [outer = 0x0] [url = http://mochi.test:8888/]
05:24:20 INFO - GECKO(894) | --DOMWINDOW == 10 (0x11eca9800) [pid = 899] [serial = 146] [outer = 0x0] [url = http://mochi.test:8888/]
05:24:20 INFO - GECKO(894) | --DOMWINDOW == 9 (0x11ed71c00) [pid = 899] [serial = 151] [outer = 0x0] [url = about:blank]
05:24:20 INFO - GECKO(894) | --DOMWINDOW == 8 (0x11ecac400) [pid = 899] [serial = 148] [outer = 0x0] [url = about:blank]
05:24:20 INFO - GECKO(894) | --DOMWINDOW == 7 (0x1180e4400) [pid = 899] [serial = 145] [outer = 0x0] [url = about:blank]
05:24:20 INFO - GECKO(894) | [Child 903, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:20 INFO - GECKO(894) | [Child 906, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:20 INFO - GECKO(894) | [Child 902, Main Thread] WARNING: nsAppShell::Exit() called redundantly: file /builds/worker/workspace/build/src/widget/cocoa/nsAppShell.mm, line 722
05:24:20 INFO - GECKO(894) | ###!!! [Child][MessageChannel] Error: (msgtype=0x840001,name=PSessionStorageObserver::Msg_DeleteMe) Channel closing: too late to send/recv, messages will be lost
05:24:20 INFO - GECKO(894) | [Child 904, Main Thread] WARNING: MsgDropped in ContentChild: file /builds/worker/workspace/build/src/dom/ipc/ContentChild.cpp, line 2227
05:24:20 INFO - GECKO(894) | Assertion failure: false (PSessionStorageObserverChild::SendDeleteMe()), at /builds/worker/workspace/build/src/dom/storage/StorageIPC.cpp:429
05:26:40 INFO - GECKO(894) | #01: mozilla::dom::SessionStorageObserver::~SessionStorageObserver() [dom/storage/SessionStorageObserver.cpp:31]

Comment 1

[Intermittent Failures Robot]

11 failures in 2818 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 2
• autoland: 8
• mozilla-central: 1

Platform breakdown:

• osx-10-10: 3
• windows7-32: 1
• linux64: 5
• windows10-64: 1
• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533651&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 3

[Intermittent Failures Robot]

38 failures in 2771 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 19
• mozilla-central: 13
• try: 1
• mozilla-inbound: 5

Platform breakdown:

• linux64: 12
• linux64-qr: 1
• osx-10-10: 12
• windows10-64-ccov: 4
• linux32: 3
• windows10-64: 2
• windows7-32: 3
• linux64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533651&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 4

[Andrei Ciure[:aciure]]

There are 32 failures associated to this bug in the last 7 days. These are occurring on all platform debug builds.

:overholt can you take a look?

Comment 5

[Intermittent Failures Robot]

38 failures in 2561 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 7
• autoland: 24
• mozilla-central: 6
• mozilla-beta: 1

Platform breakdown:

• osx-10-10: 13
• linux32: 7
• linux64: 8
• linux64-qr: 4
• windows10-64: 3
• windows10-64-ccov: 1
• windows7-32: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533651&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 6

[Jan Varga [:janv]]

Attachment: Bug 1533651 - Don't check result of PSessionStorageObserverChild::SendDeleteMe; r=asuth

Comment 7

[Andrew Sutherland [:asuth] (he/him)]

I r-'ed the patch because I think we want a more fundamental cleanup fix first. Specifically, windows already have an explicit cleanup time in nsGlobalWindowInner::FreeInnerObjects that should be invoked while PContent is still alive and all the tabs are explicitly being closed. This is where we explicitly call DisconnectEventTargetObjects() and IndexedDB gets cleared and everything.

For consistency with https://searchfox.org/mozilla-central/rev/2c912888e3b7ae4baf161d98d7a01434f31830aa/dom/base/nsGlobalWindowInner.cpp#1592-1594 I think we want to add clearing of mLocalStorage and mSessionStorage at https://searchfox.org/mozilla-central/rev/2c912888e3b7ae4baf161d98d7a01434f31830aa/dom/base/nsGlobalWindowInner.cpp#1233.

Going on previous memory, I think this might have been something that previously was cleared, but it was happening on the outer window or something like that. (LocalStorage was previously living on the outer window for unclear reasons.)

Setting needinfo to have a flag on this.

Comment 8

[Jan Varga [:janv]]

Yeah, that sounds reasonable. I'm surprised that we don't clear it already in that method.

Comment 9

[Jan Varga [:janv]]

Attachment: Bug 1533651 - Clear LocalStorage and SessionStorage in nsGlobalWindowInner::FreeInnerObjects; r=asuth

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/a36496b378c7
Clear LocalStorage and SessionStorage in nsGlobalWindowInner::FreeInnerObjects; r=asuth

Comment 11

[Daniel Varga [:dvarga]]

Backed out 2 changesets (Bug 1534211, Bug 1533651) for xpcshell failure at devtools/client/responsive.html/test/unit/test_change_device.js on a CLOSED TREE
and assertion failure: GetPersistenceFile - C:\Users\task_1553642342\AppData\Local\Temp\gv_measurements.jsonAssertion failure: gBaseDirPath, at z:/build/build/src/dom/quota/ActorsParent.cpp:2548

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=236245966&revision=00f84bd04433fb09075f32c578fda35a28394965

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=236243703&repo=mozilla-inbound&lineNumber=864
and
https://treeherder.mozilla.org/logviewer.html#?job_id=236243720&repo=mozilla-inbound

23:43:32 INFO - **** TELEMETRY: PersistenceThreadPersist
23:43:32 INFO - **** TELEMETRY: GetPersistenceFile - C:\Users\task_1553642342\AppData\Local\Temp\gv_measurements.jsonAssertion failure: gBaseDirPath, at z:/build/build/src/dom/quota/ActorsParent.cpp:2548
23:43:43 INFO - #01: nsObserverList::NotifyObservers(nsISupports *,char const *,char16_t const *) [xpcom/ds/nsObserverList.cpp:65]
23:43:43 INFO - #02: nsObserverService::NotifyObservers(nsISupports *,char const *,char16_t const *) [xpcom/ds/nsObserverService.cpp:296]
23:43:43 INFO - #03: ScopedXPCOM::~ScopedXPCOM() [xpcom/tests/TestHarness.h:106]
23:43:43 INFO - #04: mozilla::RunGTestFunc(int *,char * *) [testing/gtest/mozilla/GTestRunner.cpp:112]
23:43:43 INFO - #05: XREMain::XRE_mainStartup(bool *) [toolkit/xre/nsAppRunner.cpp:3769]
23:43:43 INFO - #06: XREMain::XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [toolkit/xre/nsAppRunner.cpp:4714]
23:43:43 INFO - #07: XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [toolkit/xre/nsAppRunner.cpp:4811]
23:43:43 INFO - #08: mozilla::BootstrapImpl::XRE_main(int,char * * const,mozilla::BootstrapConfig const &) [toolkit/xre/Bootstrap.cpp:45]
23:43:45 INFO - #09: NS_internal_main(int,char * *,char * *) [browser/app/nsBrowserApp.cpp:291]
23:43:45 INFO - #10: wmain [toolkit/xre/nsWindowsWMain.cpp:131]
23:43:45 INFO - #11: static int __scrt_common_main_seh() [f:/dd/vctools/crt/vcstartup/src/startup/exe_common.inl:288]
23:43:45 INFO - #12: kernel32.dll + 0x53c45
23:43:45 INFO - #13: ntdll.dll + 0x637f5
23:43:45 INFO - #14: ntdll.dll + 0x637c8
23:43:45 INFO - mozcrash INFO | Copy/paste: Z:\task_1553642342\build\win32-minidump_stackwalk.exe Z:\task_1553642342\build\tests\gtest\57197426-39f5-4862-8060-43bc108d00a1.dmp Z:\task_1553642342\build\symbols
23:44:05 INFO - mozcrash INFO | Saved minidump as Z:\task_1553642342\build\blobber_upload_dir\57197426-39f5-4862-8060-43bc108d00a1.dmp
23:44:05 INFO - mozcrash INFO | Saved app info as Z:\task_1553642342\build\blobber_upload_dir\57197426-39f5-4862-8060-43bc108d00a1.extra
23:44:05 WARNING - PROCESS-CRASH | gtest | application crashed [@ mozilla::dom::quota::QuotaManager::Observer::Observe(nsISupports *,char const *,char16_t const *)]
23:44:05 INFO - Crash dump filename: Z:\task_1553642342\build\tests\gtest\57197426-39f5-4862-8060-43bc108d00a1.dmp
23:44:05 INFO - Operating system: Windows NT
23:44:05 INFO - 6.1.7601 Service Pack 1
23:44:05 INFO - CPU: x86
23:44:05 INFO - GenuineIntel family 6 model 63 stepping 2
23:44:05 INFO - 8 CPUs
23:44:05 INFO - GPU: UNKNOWN
23:44:05 INFO - Crash reason: EXCEPTION_BREAKPOINT
23:44:05 INFO - Crash address: 0x563190a8
23:44:05 INFO - Process uptime: 717 seconds

Comment 12

[Jan Varga [:janv]]

So it seems that the clearing of mSessionStorage doesn't help with eliminating of the SendDeleteMe failure:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=226346436d75c8a63b4f3ffce05268514f5ca6d6&selectedJob=236321005

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b9b1b4d013f5
Clear LocalStorage and SessionStorage in nsGlobalWindowInner::FreeInnerObjects; r=asuth

Comment 14

[Andrei Ciure[:aciure]]

https://hg.mozilla.org/mozilla-central/rev/b9b1b4d013f5

Comment 15

[Andrew Sutherland [:asuth] (he/him)]

Ah, right, nsDocShell::mSessionStorageManager is a separate thing, and it seems to never be cleaned up.

:smaug, can we clear mSessionStorageManager in nsDocShell::Destroy? Maybe around where we clear mSessionHistory[1] because (hand-waving) they're related?

1: https://searchfox.org/mozilla-central/rev/3d469329a42644b41e66944d8da22d78d8c0f5ec/docshell/base/nsDocShell.cpp#4980

Comment 16

[Peter Van der Beken [:peterv]]

https://hg.mozilla.org/projects/ash/rev/a36496b378c79fea11d9d27377936ab8ffaefdb0
Bug 1533651 - Clear LocalStorage and SessionStorage in nsGlobalWindowInner::FreeInnerObjects; r=asuth

Comment 17

[Olli Pettay [:smaug][[e-mail]]]

Does that work well in case JS keeps Window object alive for a closed iframe and tries to use sessionstorage?

Comment 18

[Andrew Sutherland [:asuth] (he/him)]

That is a good question. Needinfo-ing myself for investigation/follow-up, but in the meantime I'll approve Jan's fix.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ad39ab246fc0
Don't check result of PSessionStorageObserverChild::SendDeleteMe; r=asuth

Comment 20

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/ad39ab246fc0

Comment 21

[Intermittent Failures Robot]

41 failures in 3910 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 16
• mozilla-central: 9
• mozilla-inbound: 14
• try: 1
• mozilla-beta: 1

Platform breakdown:

• linux64: 8
• linux64-qr: 5
• osx-10-10: 13
• windows10-64: 7
• windows7-32: 3
• linux32: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533651&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 22

[Jan Varga [:janv]]

Looks good, no intermittent failures after patch landing on m-i and m-c.

Comment 23

[Jan Varga [:janv]]

This probably doesn't need a beta uplift, MOZ_ALWAYS_TRUE only asserts in debug builds.

Comment 24

[Intermittent Failures Robot]

2 failures in 2843 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• osx-10-10: 1
• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533651&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 25

[Andrew Sutherland [:asuth] (he/him)]

(mSessionStorage has since moved onto the BrowsingContext and in particular uses the Top() context's mSessionStorageManager. This lifecycle setup is better in all kinds of ways now.)