Closed Bug 1341688 Opened 6 years ago Closed 4 years ago

Memory leak in IndexedDB web-platform-tests

Categories

(Core :: Storage: IndexedDB, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla65
Tracking Status
firefox-esr60 --- wontfix
firefox63 --- wontfix
firefox64 --- wontfix
firefox65 --- fixed

People

(Reporter: jgraham, Assigned: bzbarsky)

References

Details

(Keywords: memory-leak)

Attachments

(1 file)

https://treeherder.mozilla.org/logviewer.html#?job_id=79384301&repo=try&lineNumber=5445

[task 2017-02-22T15:22:19.204867Z] 15:22:19     INFO - PROCESS | 876 | Leaked URLs:
[task 2017-02-22T15:22:19.208218Z] 15:22:19     INFO - PROCESS | 876 |   file:///home/worker/workspace/build/application/firefox/omni.ja
[task 2017-02-22T15:22:19.208880Z] 15:22:19     INFO - PROCESS | 876 |   chrome://pluginproblem/content/pluginProblemBinding.css
[task 2017-02-22T15:22:19.209732Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/counterstyles.css
[task 2017-02-22T15:22:19.210635Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/html.css
[task 2017-02-22T15:22:19.211376Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/minimal-xul.css
[task 2017-02-22T15:22:19.212072Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/quirk.css
[task 2017-02-22T15:22:19.212719Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre/res/svg.css
[task 2017-02-22T15:22:19.213364Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/xul.css
[task 2017-02-22T15:22:19.214067Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/skin/scrollbars.css
[task 2017-02-22T15:22:19.214355Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/number-control.css
[task 2017-02-22T15:22:19.214806Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/forms.css
[task 2017-02-22T15:22:19.215499Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/noscript.css
[task 2017-02-22T15:22:19.215736Z] 15:22:19     INFO - PROCESS | 876 |   resource://gre-resources/ua.css
[task 2017-02-22T15:22:19.216001Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#scrollbar
[task 2017-02-22T15:22:19.216484Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml
[task 2017-02-22T15:22:19.216877Z] 15:22:19     INFO - PROCESS | 876 |   x:///chrome/toolkit/content/global/bindings/scrollbar.xml
[task 2017-02-22T15:22:19.217570Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml
[task 2017-02-22T15:22:19.217789Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#thumb
[task 2017-02-22T15:22:19.218007Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml
[task 2017-02-22T15:22:19.218477Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#scrollbar-base
[task 2017-02-22T15:22:19.218850Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#scrollbar
[task 2017-02-22T15:22:19.219325Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#scrollbar-base
[task 2017-02-22T15:22:19.219693Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#scrollbar-base
[task 2017-02-22T15:22:19.220475Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/skin/scrollbar/slider.gif
[task 2017-02-22T15:22:19.220577Z] 15:22:19     INFO - PROCESS | 876 |   chrome://global/content/bindings/scrollbar.xml#thumb
[task 2017-02-22T15:22:19.220930Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/testharness_runner.html
[task 2017-02-22T15:22:19.221392Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/testharness_runner.html
[task 2017-02-22T15:22:19.221767Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/testharness_runner.html
[task 2017-02-22T15:22:19.222215Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/IndexedDB/idbversionchangeevent.htm
[task 2017-02-22T15:22:19.222604Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/testharness_runner.html
[task 2017-02-22T15:22:19.223055Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/IndexedDB/idbversionchangeevent.htm
[task 2017-02-22T15:22:19.223708Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/IndexedDB/idbversionchangeevent.htm
[task 2017-02-22T15:22:19.224101Z] 15:22:19     INFO - PROCESS | 876 |   http://dvcs.w3.org/hg/IndexedDB/raw-file/tip/Overview.html#versionchange--transaction-steps
[task 2017-02-22T15:22:19.224416Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/resources/testharness.js
[task 2017-02-22T15:22:19.224602Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/resources/testharnessreport.js
[task 2017-02-22T15:22:19.224940Z] 15:22:19     INFO - PROCESS | 876 |   http://web-platform.test:8000/IndexedDB/support.js
[task 2017-02-22T15:22:19.225455Z] 15:22:19     INFO - PROCESS | 876 | [Child 940] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file /home/worker/workspace/build/src/xpcom/base/nsTraceRefcnt.cpp, line 172
[task 2017-02-22T15:22:19.225839Z] 15:22:19     INFO - PROCESS | 876 | nsStringStats
[task 2017-02-22T15:22:19.226237Z] 15:22:19     INFO - PROCESS | 876 |  => mAllocCount:         593307
[task 2017-02-22T15:22:19.226848Z] 15:22:19     INFO - PROCESS | 876 |  => mReallocCount:        21194
[task 2017-02-22T15:22:19.227024Z] 15:22:19     INFO - PROCESS | 876 |  => mFreeCount:          590106  --  LEAKED 3201 !!!
[task 2017-02-22T15:22:19.227299Z] 15:22:19     INFO - PROCESS | 876 |  => mShareCount:         837487
[task 2017-02-22T15:22:19.227689Z] 15:22:19     INFO - PROCESS | 876 |  => mAdoptCount:         126288
[task 2017-02-22T15:22:19.228091Z] 15:22:19     INFO - PROCESS | 876 |  => mAdoptFreeCount:     126288
[task 2017-02-22T15:22:19.228486Z] 15:22:19     INFO - PROCESS | 876 |  => Process ID: 940, Thread ID: 139689904764736
[task 2017-02-22T15:22:19.235612Z] 15:22:19     INFO - PROCESS | 876 | nsStringStats
[task 2017-02-22T15:22:19.236474Z] 15:22:19     INFO - PROCESS | 876 |  => mAllocCount:        1150911
[task 2017-02-22T15:22:19.237266Z] 15:22:19     INFO - PROCESS | 876 |  => mReallocCount:       191205
[task 2017-02-22T15:22:19.237600Z] 15:22:19     INFO - PROCESS | 876 |  => mFreeCount:         1150911
[task 2017-02-22T15:22:19.238305Z] 15:22:19     INFO - PROCESS | 876 |  => mShareCount:         916786
[task 2017-02-22T15:22:19.238562Z] 15:22:19     INFO - PROCESS | 876 |  => mAdoptCount:          33980
[task 2017-02-22T15:22:19.238906Z] 15:22:19     INFO - PROCESS | 876 |  => mAdoptFreeCount:      33980
[task 2017-02-22T15:22:19.239182Z] 15:22:19     INFO - PROCESS | 876 |  => Process ID: 876, Thread ID: 140540397332288
[task 2017-02-22T15:22:19.280018Z] 15:22:19     INFO - Browser exited with return code 0
[task 2017-02-22
Andrea, can you take a look?  If not, please let me know and I will.
Flags: needinfo?(amarchesini)
I don't have time right now. But maybe next week I could some tests. I leave the NI open.
Bevis has been looking at some test issues recently (mochitests and not WPT).
Flags: needinfo?(btseng)
I haven't found too much clue yet from cc/gc logs with the dumped BloatView. (Partially because it's my first time investigating memory leak in gecko. :|)
However, I've discovered following suspicious points during testing:
1. Leakage not only happens in this single IDB test but all the IDB test in wpt I've tried so far.
2. There won't be leakage if tested manually by 
   - Setting up the wpt-server and run ./serve
   - XPCOM_MEM_BLOAT_LOG=1 ./mach run 
   - Go to http://127.0.0.1:8000/IndexedDB/idbversionchangeevent.htm to run the test.
   - Close the browser to trigger to start the shutdown procedure. (No leak according to the bloat view result)
3. A leakage always be caught in parent process even if you run a simple wpt test like: 
> async_test(t => {
>   t.done();
> }, 'Dummy WPT test');
And the bloat view in parent looks like this:
TEST-INFO | leakcheck | default process: leaked 1 CancelableRunnable
TEST-INFO | leakcheck | default process: leaked 3 Mutex
TEST-INFO | leakcheck | default process: leaked 1 PollableEvent
TEST-INFO | leakcheck | default process: leaked 1 ReentrantMonitor
TEST-INFO | leakcheck | default process: leaked 1 Runnable
TEST-INFO | leakcheck | default process: leaked 1 nsAStreamCopier
TEST-INFO | leakcheck | default process: leaked 1 nsPipe
TEST-INFO | leakcheck | default process: leaked 1 nsPipeInputStream
TEST-INFO | leakcheck | default process: leaked 1 nsSocketTransport
TEST-INFO | leakcheck | default process: leaked 1 nsSocketTransportService
TEST-INFO | leakcheck | default process: leaked 1 nsStringBuffer
TEST-INFO | leakcheck | default process: leaked 1 nsTArray_base

Note: I enable the leak check in ./mach web-platform-tests by applying the patches in bug 1333114.
Flags: needinfo?(btseng)
Blocks: 1352355
No longer blocks: 1333114
Flags: needinfo?(amarchesini)
I'm unable to run the tests now to see if this is still happening, but it's worth noting that we changed how IDBDatabase responds to DOMEventTargetHelper::DisconnectFromOwner in bug 1450266 to invoke InvalidateInternal()[1] so it's possible that this is no longer a problem.  Specifically, before that change, IndexedDB would continue to attempt to fire events on the page after the page was dead.  These would result in errors that would then abort the transaction and unwind everything, but after the point you'd expect things to have terminated, which could in turn lead to leaks, etc.

1: https://searchfox.org/mozilla-central/rev/d850d799a0009f851b5535580e0a8b4bb2c591d7/dom/indexedDB/IDBDatabase.cpp#1206
I can reproduce this on a debug build from  m-c from today.  Simply doing:

  mach wpt IndexedDB/idbversionchangeevent.htm

shows leaks.  If I run this with --no-pause-after-test the leaks disappear, by the way, for extra fun.
OK, so we're definitely leaking an IDBDatabase.  It's being kept alive by its JS wrapper.  That JS wrapper is being kept alive via mAnonymousGlobalScopes[i] and then some maps and sandboxes and proxies and a Window by a TabChildBase.

What's keeping alive that TabChildBase?  An IDBFactory.  What's keeping that alive?  The IDBDatabase.

So basically, any time a page stores a reference to an IDBDatabase or IDBFactory at global scope, we leak.

IDBFactory is CCed, but doesn't actually CC all the things it has strong refs to.  Patch coming up.
Assignee: nobody → bzbarsky
Pushed by bzbarsky@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6416601a25e4
Fix IDB leaks due to IDBFactory not declaring all its outgoing edges to the cycle collector.  r=asuth
https://hg.mozilla.org/mozilla-central/rev/6416601a25e4
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Component: web-platform-tests → DOM: IndexedDB
Product: Testing → Core
You need to log in before you can comment on or make changes to this bug.