[B2G][WebRTC] Serious frame dropping when enabling HW H.264 video in real-time mode if the decoder has long delays

RESOLVED FIXED in Firefox 32

Status

()

defect
P2
normal
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: jhlin, Assigned: jesup)

Tracking

Trunk
mozilla33
ARM
Gonk (Firefox OS)
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox32 fixed, firefox33 fixed, b2g-v2.0 fixed, b2g-v2.1 fixed)

Details

(Whiteboard: [s=fx32, p=0])

Attachments

(1 attachment)

Apply patches in bug 911046 and start a H.264 video call. The frame rate is low.
This symptom won't occur if sender frame dropper is disabled.
Summary: [B2G] Serious frame dropping when enabling HW H.264 video in real-time mode. → [B2G][WebRTC] Serious frame dropping when enabling HW H.264 video in real-time mode.
John -- Are we still seeing this problem?
Flags: needinfo?(jolin)
Yes, especially in flame to flame video call.
It seems that long latency in OMX H.264 decoding causes WebRTC to drop decoder output frames.
Test with larger kDecoderFrameMemoryLength value (http://dxr.mozilla.org/mozilla-central/source/media/webrtc/trunk/webrtc/modules/video_coding/main/source/generic_decoder.h#25) shows much better result.
Now we need to investigate why OMX decoder latency is long (usually > 400ms in my tests).
Flags: needinfo?(jolin)
This is the ideal kind of bug for the vendor to investigate. Please make sure they can reproduce and then optimize for us.
Setting up call and emailed question on vendor plans to support low latency decode, which impacts the  2 usability bugs
Bug 989944 - [B2G][WebRTC] Serious frame dropping when enabling HW H.264 video in real-time mode.
Bug 989945 - [B2G][WebRTC] long video lag when using H.264 codec.
May be fixed with new firmware designed for KitKat.  Requested access.
Priority: -- → P2
Whiteboard: [s=fx32, p=0]
Target Milestone: --- → mozilla32
Note: the patch here wallpapers over a problem caused by excessive decoder delay in bug 989945 (an array of frames being decoded needs to be larger).  There's no reason we should modify the code permanently to handle second-plus decoder delays, so we don't want to land this patch.
This is simply due to the long decode delay in bug 989945, so dupping to that.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 989945
Reopening since after review it's safe to land the increase in buffersize for the external encoder's timestamps (effectively it's a circular-buffersize for timestamp/frame correlation).  No perf hit and virtually no mem hit to increase it.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Summary: [B2G][WebRTC] Serious frame dropping when enabling HW H.264 video in real-time mode. → [B2G][WebRTC] Serious frame dropping when enabling HW H.264 video in real-time mode if the decoder has long delays
Comment on attachment 8421325 [details] [diff] [review]
Increase decode timestamp map size to handle delayed decode

r+ (patch was by jhlin)
Attachment #8421325 - Attachment description: temp buffersize patch NOT FOR CHECKING → Increase decode timestamp map size to handle delayed decode
Attachment #8421325 - Flags: review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/b3e6f92dca28
We want this on Aurora as well, though once we get kk-based 8x10 builds it should become moot
Target Milestone: mozilla32 → mozilla33
Comment on attachment 8421325 [details] [diff] [review]
Increase decode timestamp map size to handle delayed decode

Approval Request Comment
[Feature/regressing bug #]: N/A

[User impact if declined]: Makes B2G 8x10 H.264 decode stall on builds without new DSP code that will only be available in KK-based builds.

[Describe test coverage new/current, TBPL]: Just landed on inbound; used extensively for months on Flame H.264 testing.

[Risks and why]: very low risk - simply increases the default size of a circular buffer used to match decoded frames with the inputs.  No perf downsize

[String/UUID change made/needed]: none
Attachment #8421325 - Flags: approval-mozilla-aurora?
Comment on attachment 8421325 [details] [diff] [review]
Increase decode timestamp map size to handle delayed decode

Looks very safe. Aurora+ even though this isn't on m-c yet.
Attachment #8421325 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
https://hg.mozilla.org/mozilla-central/rev/b3e6f92dca28
Assignee: nobody → rjesup
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.