1023125 - [Dolphin] H264 OMX Encoder does not support prepend-sps-pps-to-idr-frames (needed by WebrtcOMXH264VideoEncoder)

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[Andreas Pehrson [:pehrsons]]

Steps to reproduce:
1. On your dolphin, go to http://mozilla.github.io/webrtc-landing/pc_test.html
2. Select one-way call and click Start!
3. Wait until the ice connection has been started. If you get "ICE connection failed" in logcat, start over. I had a success rate of ICE of ~30% when testing this, not sure why it was so low.

Expected result: OMX Encoder configured successfully and starts receiving frames.

Actual result: When configuring the encoder, logcat spits out the following error:
06-10 05:22:31.137  3163  3285 I Gecko   :  WebrtcOMXH264VideoCodec :: WebrtcOMXH264VideoEncoder:0xb3082240 configuring encoder 240x320 @ 30 fps, rate 300 kb#
06-10 05:22:31.137   100   826 I SprdOMXComponent: SetParameterWrapper
06-10 05:22:31.137   100   826 I SPRDAVCEncoder: this: b750d8f0, func: internalSetParameter
06-10 05:22:31.137   100  1122 I SprdOMXComponent: GetExtensionIndexWrapper
06-10 05:22:31.137   100  1122 I SPRDAVCEncoder: this: b750d8f0, func: getExtensionIndex
06-10 05:22:31.137   100  1122 I SprdOMXComponent: this: b750d8f0, func: getExtensionIndex
06-10 05:22:31.137  3163  3296 E ACodec  : Encoder could not be configured to emit SPS/PPS before IDR frames. (err -2147483648)
06-10 05:22:31.137  3163  3296 E ACodec  : [OMX.sprd.h264.encoder] configureCodec returning error -2147483648                                                 
06-10 05:22:31.137  3163  3295 E MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -2147483648)

For reference, the error in the log happens after WebrtcOMXH264VideoEncoder::Encode calls MediaCodec::ConfigureDirect.

It does not happen when "format->setInt32("prepend-sps-pps-to-idr-frames", 1);" is commented out here: http://dxr.mozilla.org/mozilla-central/source/media/webrtc/signaling/src/media-conduit/WebrtcOMXH264VideoCodec.cpp#810

I see this could be fixed in either the codec or in gecko, but not knowing anything about the implications of "prepend-sps-pps-to-idr-frames" I'll leave that to the experts.

Comment 1

[Kai-Zhen Li [:seinlin][:kli]]

Sam, need your help the check this in codec.

Comment 2

[Kai-Zhen Li [:seinlin][:kli]]

John, can you have a look to this bug?

Comment 3

[John Lin [:jhlin][:jolin]]

(In reply to Kai-Zhen Li from comment #2)
> John, can you have a look to this bug?

 Randell, looks like SPRD H.264 encoder doesn't support prepending SPS/PPS before I-frame and reports error if it's requested. IIRC, QCT 8x10 doesn't support that either, and we already have workaround for it in landed code. Do you think keeping this setting is still necessary?

 BTW, I checked the AOSP code and found Google's workaround for this [1], but IMHO that makes sense only when there are just few encoders which doesn't have this feature. Maybe most encoders on high-end devices do support it?

[1] http://androidxref.com/4.4_r1/xref/frameworks/av/media/libstagefright/wifi-display/source/Converter.cpp#220

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

It may be worse than that: the 8x10 would fail with that "wifi" code, since it silently ignores the request to prepend SPS/PPS.  And that's not the only reason configure can fail, so once you go beyond 2 or 3 things like this it gets very, very ugly.

We can simply remove it.... but if the codec supports it we want it.  The stagefright interface (and even worse the implementations) sucks.  Either we need QC to fix it to fail configure on the 8x10, or we need the Dolphin to not fail it (but that might cause other problems; see google's code).  Or we can remove the parameter entirely, which is sad-making, but may be simplest for now.

Comment 5

[sam.hua]

Hi,
SPRD doesn't support it now.

Comment 6

[James Zhang (Spreadtrum)]

Our Codec don't support "prepend-sps-pps-to-idr-frames" now, do you really need it?
And our codec owner asked how to print the error log, do we miss some patches?

Comment 7

[Andreas Pehrson [:pehrsons]]

I'll defer your first question to Randell, the Mozilla WebRTC owner.

If there are log prints from your parts that you don't have, I might have added them while debugging. The stuff from ACodec/MediaCodec should already be there though.

Comment 8

[Randell Jesup [:jesup] (needinfo me)]

We don't need it; it was included because we wanted it if possible.  Since other implementations we're dealing with ignore it, we have code to cache the SPS/PPS at the start of a stream and insert them on all following "naked" IDRs.

It's mildly unfortunate, but we can stop asking for SPS/PPS.

Will your codec generate them if some value within them changes, or do all such input/config changes require stop and re-open of the encoder?

Also: is constant-bitrate-with-skips supported?  (or constant-without-skips?)  And can we adjust the bitrate inbetween IFrames?  Also, if the input framerate is variable (i.e. config for 30fps but camera changes rate from 30 to 20), does the encoder adjust the number of bits-per-frame based on the apparent rate from the timestamps?  I.e. if input drops to 20fps, will it use the configured bitrate, or 2/3 the configured rate?

Comment 9

[Alan]

Hi Randell,

For you first query, if such input/config changes we need stop and re-open our encoder.

For the CBT question, it will be replied later by another engineer.

Comment 10

[James Zhang (Spreadtrum)]

Alan, any process here?

Comment 11

[James Zhang (Spreadtrum)]

Owner said we don't support this feature now.
If you need workaround patch, you should confirm the resolution and rate.

Comment 12

[BMO Automation]

Firefox OS is not being worked on