846071 - "OOM abort in test_peerConnection_basicVideo.html (preceded by assertions in test_peerConnection_basicAudioVideo.html ?)

Status: RESOLVED INCOMPLETE

(Core :: WebRTC: Networking, defect, P5)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=20159969&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitest-3 on 2013-02-27 15:24:35 PST for push 1b338f7f9fcc
slave: talos-r3-fed-088

shows a new intermittent failure

first, and not necessarily related, is this:
15:29:23     INFO -  339 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/media/tests/mochitest/test_peerConnection_basicAudioVideo.html | Assertion count 6 is greater than expected range 0-1 assertions.

and then the bit that this bug is definitely about:
15:29:23  WARNING -  TEST-UNEXPECTED-FAIL | /tests/dom/media/tests/mochitest/test_peerConnection_basicVideo.html | Exited with code 11 during test run
15:29:25     INFO -  PROCESS-CRASH | /tests/dom/media/tests/mochitest/test_peerConnection_basicVideo.html | application crashed [@ mozalloc_abort(char const*)]
15:30:50    ERROR - Return code: 1


The (possibly related) assertions are multiple occurrences of:
###!!! ASSERTION: Buffer underran: 'bufferEnd >= mCurrentTime', file ../../../content/media/MediaStreamGraph.cpp, line 392

The abort doesn't have a useful stack beyond:

15:29:25     INFO -  Operating system: Linux
15:29:25     INFO -                    0.0.0 Linux 2.6.31.5-127.fc12.i686.PAE #1 SMP Sat Nov 7 21:25:57 EST 2009 i686
15:29:25     INFO -  CPU: x86
15:29:25     INFO -       GenuineIntel family 6 model 23 stepping 10
15:29:25     INFO -       2 CPUs
15:29:25     INFO -  Crash reason:  SIGSEGV
15:29:25     INFO -  Crash address: 0x0
15:29:25     INFO -  Thread 22 (crashed)
15:29:25     INFO -   0  libmozalloc.so!mozalloc_abort(char const*) [mozalloc_abort.cpp:1b338f7f9fcc : 30 + 0x0]
15:29:25     INFO -      eip = 0x001f9f0d   esp = 0xa6afefb0   ebp = 0xa6afefc8   ebx = 0x001fb120
15:29:25     INFO -      esi = 0x00c59844   edi = 0x00000000   eax = 0x0000000a   ecx = 0x00000001
15:29:25     INFO -      edx = 0x00c5a32c   efl = 0x00210246
15:29:25     INFO -      Found by: given as instruction pointer in context
15:29:25     INFO -   1  libmozalloc.so!mozalloc_handle_oom(unsigned int) [mozalloc_oom.cpp:1b338f7f9fcc : 27 + 0xe]
15:29:25     INFO -      eip = 0x001f9f52   esp = 0xa6afefd0   ebp = 0xa6afefe8
15:29:25     INFO -      Found by: previous frame's frame pointer
15:29:25     INFO -   2  libmozalloc.so!moz_xmalloc [mozalloc.cpp:1b338f7f9fcc : 56 + 0x8]
15:29:25     INFO -      eip = 0x001f9b8d   esp = 0xa6afeff0   ebp = 0xa6aff008
15:29:25     INFO -      Found by: previous frame's frame pointer

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to David Baron [:dbaron] (don't cc:, use needinfo? instead) from comment #0)
> The (possibly related) assertions are multiple occurrences of:
> ###!!! ASSERTION: Buffer underran: 'bufferEnd >= mCurrentTime', file
> ../../../content/media/MediaStreamGraph.cpp, line 392

This is bug 839650 which we might want to set as dependency.

David, can you help us to understand why this is an OOM abort?

Comment 2

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=20163885&tree=Mozilla-Inbound
Rev3 Fedora 12 mozilla-inbound debug test mochitest-3 on 2013-02-27 17:08:33 PST for push 43a54aaca03c
slave: talos-r3-fed-006


hit the same "Buffer underran" assertion, but in that test rather than the previous, and then:

17:12:55     INFO -  out of memory


(In reply to Henrik Skupin (:whimboo) from comment #1)
> David, can you help us to understand why this is an OOM abort?

no idea what you mean

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to David Baron [:dbaron] (don't cc:, use needinfo? instead) from comment #2)
> (In reply to Henrik Skupin (:whimboo) from comment #1)
> > David, can you help us to understand why this is an OOM abort?
> 
> no idea what you mean

I have checked the referenced logs and I found the 'INFO -  out of memory' line in it. So my question has been automatically be answered.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Clint, lately you mentioned an OOM issue when you run the webrtc tests. Can you reproduce that one, and is it related?

Comment 6

[Eric Rescorla (:ekr)]

Thanks for catching the dup.

Comment 7

[Randell Jesup [:jesup] (needinfo me)]

Looking at the log from the dup, I see basicVideo ending, but it continues to send video frames, implying that sipcc failed to shut down the call or was somehow delayed.

Pretty rare, though we've had like 3 instances total.

-> abr

Comment 8

[Adam Roach [:abr]]

I'm keeping my eye on this and waiting for it to surface again. Back on 2/28 (right after the most recent report), I added instrumentation to the relatively opaque "out of memory" log message to determine whether this is a true out-of-memory condition (which I find extremely unlikely) or whether it is a request somewhere to allocate a bogus amount of memory in a single block. See Bug 846368.

Given, however, that the last reported citing of this bug in the wild was several weeks ago, I'm pushing it down to P5 and letting it simmer on the back burner to see if it shows up again. If it does, the extra instrumentation in the OOM message should provide hints as to where to look next.

Comment 9

[Jason Smith [:jsmith]]

Old bug and not actionable. let's close this for now as incomplete - if it reproduces again, we can reopen and do more analysis.