Closed Bug 1565501 Opened 4 months ago Closed 4 months ago

Crash in [@ OOM | large | NS_ABORT_OOM | nsTArray_base<T>::EnsureCapacity<T> | nsTArray_Impl<T>::AppendElements<T> | mozilla::MediaSpan::Append]

Categories

(Core :: Audio/Video: Playback, defect, P2, critical)

69 Branch
All
Windows
defect

Tracking

()

RESOLVED FIXED
mozilla70
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox68 --- unaffected
firefox69 --- fixed
firefox70 --- fixed

People

(Reporter: philipp, Assigned: alwu)

References

(Regression)

Details

(Keywords: crash, regression)

Crash Data

Attachments

(1 file)

This bug is for crash report bp-8d12f31c-4cdd-4265-8290-ca7f70190710.

Top 10 frames of crashing thread:

0 xul.dll NS_ABORT_OOM xpcom/base/nsDebugImpl.cpp:604
1 xul.dll nsTArray_base<nsTArrayInfallibleAllocator, nsTArray_CopyWithMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator> xpcom/ds/nsTArray-inl.h:136
2 xul.dll nsTArray_Impl<unsigned char, nsTArrayInfallibleAllocator>::AppendElements<unsigned char, nsTArrayInfallibleAllocator> xpcom/ds/nsTArray.h:2355
3 xul.dll bool mozilla::MediaSpan::Append dom/media/MediaSpan.h:89
4 xul.dll void mozilla::TrackBuffersManager::ProcessTasks dom/media/mediasource/TrackBuffersManager.cpp:217
5 xul.dll void mozilla::TrackBuffersManager::QueueTask dom/media/mediasource/TrackBuffersManager.cpp:170
6 xul.dll class RefPtr<mozilla::MozPromise<mozilla::Pair<bool, mozilla::SourceBufferAttributes>, mozilla::MediaResult, 1> > mozilla::TrackBuffersManager::DoAppendData dom/media/mediasource/TrackBuffersManager.cpp:142
7 xul.dll nsresult mozilla::detail::ProxyRunnable<mozilla::MozPromise<mozilla::Pair<bool, mozilla::SourceBufferAttributes>, mozilla::MediaResult, 1>, RefPtr<mozilla::MozPromise<mozilla::Pair<bool, mozilla::SourceBufferAttributes>, mozilla::MediaResult, 1> >  xpcom/threads/MozPromise.h:1313
8 xul.dll nsresult mozilla::TaskQueue::Runner::Run xpcom/threads/TaskQueue.cpp:199
9 xul.dll nsresult nsThreadPool::Run xpcom/threads/nsThreadPool.cpp:244

these out of memory crash signatures are newly showing up since firefox 69 - looking related to the changes in bug 1558364.

in the extreme case of this particular report, firefox attempted to allocate over 2GB of memory.

Jean-Yves do you know who would be a good candidate to look into this regression caused by one of cpearce last patches?

Flags: needinfo?(jyavenard)
Priority: -- → P2

Not sure.
This needs to be changed to use fallible memory allocation, like everything else in the media/mse stack

Flags: needinfo?(jyavenard)

Not sure.
This needs to be changed to use fallible memory allocation, like everything else in the media/mse stack

Assignee: nobody → alwu

I found that the amount of hidden bytes, which we would hide by using RemoveFront(), are quite large when watching video in Youtbe. I can see the percentage varied from 0~100. So beside using fallible memory allocation, maybe we should also think about reducing the amount of hidden bytes in MediaSpan to reduce the memory usage?

Here are some debug logs I printed for the video [1], in this log the highest hidden bytes is 3728316 bytes.

[1] https://www.youtube.com/watch?v=mzIsfSsZh7I

DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 135601 bytes, hide 10362 bytes, pecentage=7.641537
DD | - in resource queue 0x7f80bdbcc4a0, visible 154604 bytes, hidden 556907 bytes, hidden percentage 78.271034
DD | - in resource queue 0x7f80b4562120, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 125239 bytes, hide 41132 bytes, pecentage=32.842805
DD | - in resource queue 0x7f80bdbcc4a0, visible 172484 bytes, hidden 482658 bytes, hidden percentage 73.672272
DD | - in resource queue 0x7f80b45621c0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 84107 bytes, hide 16631 bytes, pecentage=19.773622
DD | - in resource queue 0x7f80bdbcc4a0, visible 57763 bytes, hidden 349040 bytes, hidden percentage 85.800743
DD | - in resource queue 0x7f80b4562260, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 67476 bytes, hide 6971 bytes, pecentage=10.331081
DD | - in resource queue 0x7f80bdbcc4a0, visible 23602 bytes, hidden 383201 bytes, hidden percentage 94.198174
DD | - in resource queue 0x7f80b45629e0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 60505 bytes, hide 39241 bytes, pecentage=64.855797
DD | - in resource queue 0x7f80bdbcc4a0, visible 46212 bytes, hidden 360591 bytes, hidden percentage 88.640198
DD | - in resource queue 0x7f80b4562b20, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 21264 bytes, hide 21264 bytes, pecentage=100.000000
DD | - in resource queue 0x7f80bdbcc4a0, visible 60505 bytes, hidden 346298 bytes, hidden percentage 85.126709
DD | - in resource queue 0x7f80b7942cc0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - in resource queue 0x7f80bdbccf40, visible 246134 bytes, hidden 2620 bytes, hidden percentage 1.053249
DD | ============ Start CodedFrameProcessing =========
DD | - in resource queue 0x7f80bdbccf40, visible 333657 bytes, hidden 2620 bytes, hidden percentage 0.779120
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 1257977 bytes, hide 16999 bytes, pecentage=1.351297
DD | - in resource queue 0x7f80bdbcc4a0, visible 38263 bytes, hidden 1490916 bytes, hidden percentage 97.497810
DD | - in resource queue 0x7f80b2aabae0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 1240978 bytes, hide 87252 bytes, pecentage=7.030906
DD | - in resource queue 0x7f80bdbcc4a0, visible 104251 bytes, hidden 2547304 bytes, hidden percentage 96.068306
DD | - in resource queue 0x7f80b2aabb80, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 1153726 bytes, hide 47381 bytes, pecentage=4.106781
DD | - in resource queue 0x7f80bdbcc4a0, visible 134633 bytes, hidden 3639298 bytes, hidden percentage 96.432556
DD | - in resource queue 0x7f80b2aabc20, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 1106345 bytes, hide 50251 bytes, pecentage=4.542073
DD | - in resource queue 0x7f80bdbcc4a0, visible 97632 bytes, hidden 3676299 bytes, hidden percentage 97.412994
DD | - in resource queue 0x7f80b2aabcc0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 1056094 bytes, hide 100119 bytes, pecentage=9.480122
DD | - in resource queue 0x7f80bdbcc4a0, visible 150370 bytes, hidden 3623561 bytes, hidden percentage 96.015564
DD | - in resource queue 0x7f80b2aabd60, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 955975 bytes, hide 30975 bytes, pecentage=3.240147
DD | - in resource queue 0x7f80bdbcc4a0, visible 131094 bytes, hidden 3642837 bytes, hidden percentage 96.526321
DD | - in resource queue 0x7f80b2aabe00, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 925000 bytes, hide 21167 bytes, pecentage=2.288324
DD | - in resource queue 0x7f80bdbcc4a0, visible 52142 bytes, hidden 3721789 bytes, hidden percentage 98.618362
DD | - in resource queue 0x7f80b2aabea0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 903833 bytes, hide 35872 bytes, pecentage=3.968875
DD | - in resource queue 0x7f80bdbcc4a0, visible 57039 bytes, hidden 3716892 bytes, hidden percentage 98.488602
DD | - in resource queue 0x7f80b2aabf40, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 867961 bytes, hide 32217 bytes, pecentage=3.711803
DD | - in resource queue 0x7f80bdbcc4a0, visible 68089 bytes, hidden 3705842 bytes, hidden percentage 98.195808
DD | - in resource queue 0x7f80b2aabfe0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 835744 bytes, hide 117355 bytes, pecentage=14.041979
DD | - in resource queue 0x7f80bdbcc4a0, visible 149572 bytes, hidden 3624359 bytes, hidden percentage 96.036705
DD | - in resource queue 0x7f80b2aac080, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 718389 bytes, hide 37072 bytes, pecentage=5.160435
DD | - in resource queue 0x7f80bdbcc4a0, visible 154427 bytes, hidden 3619504 bytes, hidden percentage 95.908058
DD | - in resource queue 0x7f80b2aac120, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 681317 bytes, hide 20599 bytes, pecentage=3.023409
DD | - in resource queue 0x7f80bdbcc4a0, visible 57671 bytes, hidden 3716260 bytes, hidden percentage 98.471855
DD | - in resource queue 0x7f80b2aac1c0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 660718 bytes, hide 25016 bytes, pecentage=3.786184
DD | - in resource queue 0x7f80bdbcc4a0, visible 45615 bytes, hidden 3728316 bytes, hidden percentage 98.791313
DD | - in resource queue 0x7f80b2aac260, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 635702 bytes, hide 33485 bytes, pecentage=5.267405
DD | - in resource queue 0x7f80bdbcc4a0, visible 58501 bytes, hidden 3715430 bytes, hidden percentage 98.449867
DD | - in resource queue 0x7f80b2aac300, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 602217 bytes, hide 86449 bytes, pecentage=14.355124
DD | - in resource queue 0x7f80bdbcc4a0, visible 119934 bytes, hidden 3653997 bytes, hidden percentage 96.822044
DD | - in resource queue 0x7f80b2aac4e0, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 515768 bytes, hide 151377 bytes, pecentage=29.349824
DD | - in resource queue 0x7f80bdbcc4a0, visible 237826 bytes, hidden 3536105 bytes, hidden percentage 93.698189
DD | - in resource queue 0x7f80b2aac760, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 364391 bytes, hide 364391 bytes, pecentage=100.000000
DD | - in resource queue 0x7f80bdbcc4a0, visible 515768 bytes, hidden 3258163 bytes, hidden percentage 86.333397
DD | - in resource queue 0x7f80b2aaca80, visible 220 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 331510 bytes, hide 20499 bytes, pecentage=6.183524
DD | - in resource queue 0x7f80bdbccf40, visible 354156 bytes, hidden 313631 bytes, hidden percentage 46.965725
DD | - in resource queue 0x7f80b2aabb80, visible 266 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========
DD | ============ Start CodedFrameProcessing =========
DD | - process coded frame, buffer total size 311011 bytes, hide 156240 bytes, pecentage=50.236165
DD | - in resource queue 0x7f80bdbccf40, visible 329798 bytes, hidden 535807 bytes, hidden percentage 61.899712
DD | - in resource queue 0x7f80b2aabb80, visible 266 bytes, hidden 0 bytes, hidden percentage 0.000000
DD | ============ Completed CodedFrameProcessing =========

(In reply to Alastor Wu [:alwu] from comment #4)

I found that the amount of hidden bytes, which we would hide by using RemoveFront(), are quite large when watching video in Youtbe. I can see the percentage varied from 0~100. So beside using fallible memory allocation, maybe we should also think about reducing the amount of hidden bytes in MediaSpan to reduce the memory usage?

This is what we used to do without MediaSpan, but then you get either hundreds of tiny allocations all the time or hundreds of byte shifting.

In any case here, the issue is the same as what we had before. Except that before the allocation was fallible, and now it's not.

Use fallible memory allocation in order to avoid the OOM issue.

Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/82c5bbd69b94
use fallible memory allocation in MediaSpan. r=jya
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70

Please nominate this patch for Beta approval when you get a chance.

Flags: needinfo?(alwu)

Comment on attachment 9078849 [details]
Bug 1565501 - use fallible memory allocation in MediaSpan.

Beta/Release Uplift Approval Request

  • User impact if declined: Firefox might crash when there is no any usable memory because we used the infallible memory allocation for appending a media sample, which has not considered about the OOM.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce: no
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): If OOM happens, we would return error to stop media playback instead, which is always better than having a crash.
  • String changes made/needed: no
Flags: needinfo?(alwu)
Attachment #9078849 - Flags: approval-mozilla-beta?
Crash Signature: [@ OOM | large | NS_ABORT_OOM | nsTArray_base<T>::EnsureCapacity<T> | nsTArray_Impl<T>::AppendElements<T> | mozilla::MediaSpan::Append] [@ OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | nsTArray_base<T>::EnsureCapacity<T> | nsTArray… → [@ OOM | large | NS_ABORT_OOM | nsTArray_base<T>::EnsureCapacity<T> | nsTArray_Impl<T>::AppendElements<T> | mozilla::MediaSpan::Append] [@ OOM | large | mozalloc_abort | mozalloc_handle_oom | moz_xrealloc | nsTArray_base<T>::EnsureCapacity<T> | nsTArray_…

Comment on attachment 9078849 [details]
Bug 1565501 - use fallible memory allocation in MediaSpan.

Improves OOM handling to avoid crashing in some cases. Approved for 69.0b7.

Attachment #9078849 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
You need to log in before you can comment on or make changes to this bug.