Closed Bug 858948 Opened 12 years ago Closed 11 years ago

Intermittent Win7 M2 leakcheck | lots of bytes leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule, CalculateFrecencyFunction, ...), (AsyncStatement, AtomImpl, Attr, BackstagePass, BodyRule, ...) where leaked URL list includes imptests tests

Categories

(Core :: DOM: Core & HTML, defect)

Product:
Core
Component:
DOM: Core & HTML
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla27
Tracking Flags:
Tracking Status
firefox25 --- wontfix
firefox26 --- fixed
firefox27 --- fixed
firefox-esr24 --- wontfix

People

(Reporter: philor, Assigned: mccr8)

References

Details

(Keywords: intermittent-failure, memory-leak) 

This one's at least the third one, because there was one that I tried to pin on Ms2ger's merge from m-c and then on Aryeh's push right before it, and another one which got an innocent patch backed out.

https://tbpl.mozilla.org/php/getParsedLog.php?id=21502592&tree=Mozilla-Inbound&full=1
Rev3 WINNT 6.1 mozilla-inbound debug test mochitest-2 on 2013-04-05 16:55:47 PDT for push 9e6fe353c9b4
slave: talos-r3-w7-107

00:23:46     INFO -  Leaked URLs:
...
00:23:46     INFO -    http://es5.github.com/#x15.4.4.5
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-commonAncestorContainer.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-commonAncestorContainer.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-commonAncestorContainer.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-deleteContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-deleteContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-insertNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-insertNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-insertNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-insertNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-mutations.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-selectNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-selectNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-selectNode.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-set.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-set.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-set.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-set.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharness.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/resources/testharnessreport.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/Range-test-iframe.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/common.js
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/DOMCore/tests/approved/ayg@aryeh.name
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/?autorun=1&closeWhenDone=1&consoleLevel=INFO&totalChunks=5&thisChunk=2&chunkByDir=4&hideResultsTable=1
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/WebStorage/tests/submissions/Ms2ger/test_storage_session_index_js.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/WebStorage/tests/submissions/Ms2ger/test_storage_session_length_js.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/WebStorage/tests/submissions/Ms2ger/test_storage_session_removeitem_js.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/dom/imptests/webapps/WebStorage/tests/submissions/Ms2ger/test_storage_session_setitem_js.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
00:23:46     INFO -    http://mochi.test:8888/tests/SimpleTest/iframe-between-tests.html
...
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 1 AsyncStatement (48 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 189 AtomImpl (4536 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 15 BackstagePass (540 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 20 BodyRule (320 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 1 CalculateFrecencyFunction (12 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 4 CallbackObject (64 bytes)
00:23:54     INFO -  TEST-INFO | leakcheck | leaked 27984 Comment (1902912 bytes)
...
We aren't actually loading rel=help, over the network, are we? Just creating a something that we never use, and then leak?
Summary: Intermittent Win7 M2 leakcheck | 132186283 bytes leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule, CalculateFrecencyFunction, ...) where leaked URL list includes imptests tests → Intermittent Win7 M2 leakcheck | 44953315,132186283 bytes leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule, CalculateFrecencyFunction, ...) where leaked URL list includes imptests tests
https://tbpl.mozilla.org/php/getParsedLog.php?id=21607822&tree=Mozilla-Inbound&full=1#error0
Summary: Intermittent Win7 M2 leakcheck | 44953315,132186283 bytes leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule, CalculateFrecencyFunction, ...) where leaked URL list includes imptests tests → Intermittent Win7 M2 leakcheck | lots of bytes leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule, CalculateFrecencyFunction, ...), (AsyncStatement, AtomImpl, Attr, BackstagePass, BodyRule, ...) where leaked URL list includes imptests tests
(In reply to TinderboxPushlog Robot from comment #31)
> RyanVM
> https://tbpl.mozilla.org/php/getParsedLog.php?id=21697075&tree=Mozilla-
> Inbound
> Rev3 WINNT 6.1 mozilla-inbound debug test mochitest-2 on 2013-04-11 06:12:17
> slave: talos-r3-w7-021
> 
> 13:41:26     INFO -  8788 ERROR TEST-UNEXPECTED-FAIL |
> /tests/dom/imptests/webapps/DOMCore/tests/approved/test_exceptions.html |
> Assertion count 1 is greater than expected range 0-0 assertions.
> 13:45:26  WARNING -  TEST-UNEXPECTED-FAIL | leakcheck | 44281711 bytes
> leaked (AsyncStatement, AtomImpl, BackstagePass, BodyRule,
> CalculateFrecencyFunction, ...)

Maybe relevant?

13:41:20     INFO -  [Parent 2936] ###!!! ASSERTION: didn't call Disconnect: '!mTarget', file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/content/events/src/nsEventListenerManager.cpp, line 131
Smaug, does the assertion in comment 32 look like it could be relevant to this?
Flags: needinfo?(bugs)
I doubt.
Flags: needinfo?(bugs)
Whiteboard: [MemShrink]
Assignee: nobody → Ms2ger
Whiteboard: [MemShrink] → [MemShrink:P2]
Assignee: Ms2ger → nobody
Any other suggestions, Nick?
Flags: needinfo?(n.nethercote)
> Any other suggestions, Nick?

No, sorry.
Flags: needinfo?(n.nethercote)
Back to Jet, then!
Flags: needinfo?(bugs)
(In reply to Ryan VanderMeulen [:RyanVM] from comment #74)
> Back to Jet, then!

Jet, ping?
Andrew, any chance you can help diagnose?
Flags: needinfo?(continuation)
Assignee: nobody → continuation
Flags: needinfo?(continuation)
Flags: needinfo?(bugs)
I don't really know what to do here, but I guess I can take a look at it.  There's a lot of test harness stuff leaking.
(In reply to Andrew McCreight [:mccr8] from comment #164)
> I don't really know what to do here, but I guess I can take a look at it. 
> There's a lot of test harness stuff leaking.

Any luck so far, Andrew?
Flags: needinfo?(continuation)
no
Flags: needinfo?(continuation)
Depends on: 874258
There are a huge number of failures like this in the log:
06:05:28     INFO -  [Parent 1988] WARNING: NS_ENSURE_TRUE(newRoot) failed: file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/content/base/src/nsRange.cpp, line 1244

I'm not sure what that means.

Then we free a bunch of DOMWINDOWs, and apparently run out of memory during cycle collection:

06:07:12     INFO -  [Parent 1988] WARNING: Hash table add in GCGraphBuilder::AddNode failed: file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/xpcom/base/nsCycleCollector.cpp, line 1888

That line repeated over and over is most of the log.

Then near the end it switches to:

06:08:12     INFO -  [Parent 1988] WARNING: out of memory: file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/obj-firefox/xpcom/build/nsDeque.cpp, line 178

That cycle collection takes over a minute.

It isn't surprising that we leak when we run out of memory during a CC.  I could change the CC to try to recover from OOMs a little more, but the problem may lie earlier.  I can at least make it assert once per CC if we run out of memory, so we can get a clearer picture of when this is happening.

This looks suspicious, from earlier in the log, but maybe it is normal:
05:53:30     INFO -  [Parent 1988] WARNING: Could not get disk information from DiskSpaceWatcher: file e:/builds/moz2_slave/m-in-w32-d-0000000000000000000/build/dom/src/storage/DOMStorageIPC.cpp, line 324
It happens a number of times.  That was added in bug 861920, and is triggered when disk space is running low, it sounds like.  So things are not in a great state on the machine.
Depends on: 875304
This is almost certainly the same issue as bug 867974.  We're OOMing right after we finish the test 
  dom/imptests/webapps/DOMCore/tests/approved/test_Range-surroundContents.html
which is known to cause OOM issues on mobile.
Blocks: 875585
I'm clearing the MemShrink tag as this looks like a leak resulting from a test OOMing the browser, causing the CC to fail, which is kind of a separate issue.
Whiteboard: [MemShrink:P2]
Disabling the crummy test fixed this, it looks like.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WORKSFORME
seems its back -> https://tbpl.mozilla.org/php/getParsedLog.php?id=26148359&tree=Mozilla-Inbound
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
That's peculiar.  No CC OOMs.
something riled this leak up good
Olli, did you land something snow-white related on August 5?  I'm wondering if there's some new place to OOM that we're not tracking...
Flags: needinfo?(bugs)
I haven't landed any SW related patches on August.
Flags: needinfo?(bugs)
comment 310 also hit bug 900633, which may be relevant given the assertion there:

###!!! ASSERTION: Ran out of memory while building cycle collector graph: 'false', file e:/builds/moz2_slave/m-aurora-w32-d-000000000000000/build/xpcom/base/nsCycleCollector.cpp, line 2141
This hasn't happened on trunk for a while.  I'm calling this fixed by bug 920043.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Depends on: 920043
Resolution: --- → FIXED
Target Milestone: --- → mozilla27
Blocks: 933741
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.