Closed Bug 2082956 Opened 1 month ago Closed 28 days ago

Android with FFmpeg MediaCodec decoding reports support for high bit depth video the device cannot decode

Categories

(Core :: Audio/Video, defect)

All
Android
defect

Tracking

()

RESOLVED FIXED
160 Branch
Tracking Status
firefox160 --- fixed

People

(Reporter: aosmond, Assigned: aosmond)

References

(Blocks 4 open bugs)

Details

Attachments

(5 external files)

When decoding on Android goes through FFmpeg with MediaCodec (media.ffvpx-hw.enabled), colour depth is ignored when reporting decoder support. This shows up in navigator.mediaCapabilities.decodingInfo(): for example, 12-bit VP9 HDR (vp09.02.51.12...) is reported as powerEfficient, which makes the /_mozilla/media-capabilities/decodingInfo-api-modes.any.html?mode=supportscache WPT pass unexpectedly in bug 1981478.

Causes:

  • FFmpegDecoderModule::Supports overrides the base implementation without calling SupportsColorDepth, so MediaCodec is reported as available for VP8/VP9/AV1/H264/HEVC at any depth.
  • RemoteDecoderModule::Supports in the content process answers from the MIME type and the cached MediaCodecsSupported bits only, so it never sees colour depth either.
  • FFmpegVideoDecoder::InitMediaCodecDecoder only rejects content above 10-bit. ffvpx has no JavaVM set, so the decoder is created with AMediaCodec_createDecoderByType without considering the profile, and may not support 10-bit at all.

MediaCodec cannot decode more than 10-bit, and 10-bit support depends on the device. The Java decoder path (AndroidDecoderModule) did check this, via HardwareCodecCapabilityUtils.decodes10Bit.

Proposed fix:

  • Report 10-bit decoder support from HardwareCodecCapabilityUtils.getSupportedMimeTypesWithAccelInfo and expose it through a new PlatformMediaCodecs10BitSupported gfxVar.
  • Use it in AndroidDecoderModule, FFmpegDecoderModule, FFmpegVideoDecoder::InitMediaCodecDecoder and RemoteDecoderModule.
  • MediaCodec support is only reported for 10-bit when a 10-bit decoder exists, and never above 10-bit. VP9 and AV1 still fall back to ffvpx software decoding at any depth; H264 and HEVC have no software fallback on Android.
Depends on: 2083015

HardwareCodecCapabilityUtils.getSupportedMimeTypesWithAccelInfo now also lists
decoders which support a 10-bit profile, and the largest frame size supported
by hardware video decoders. These are exposed through the new
PlatformMediaCodecs10BitSupported gfxVar and the Platform*HwDecodeMaxSize
gfxVars for H264, HEVC, VP8, VP9 and AV1, so that both the content process and
the media processes can use them.

MediaCodec support is now only reported for 10-bit video when the device has a
10-bit decoder for the codec, and never above 10-bit. Hardware support is also
dropped when the frame is larger than any hardware decoder supports.
AndroidDecoderModule, FFmpegDecoderModule, FFmpegVideoDecoder and
RemoteDecoderModule share the same rules, so the Java and NDK decoding paths
report the same support. H264 and HEVC report software decoding when the
platform has a software decoder, and hardware decoding when it is available.

decodes10Bit has no remaining callers, and the encoder MIME types were always
requested from getSupportedMimeTypesWithAccelInfo.

Blocks: 2083164
Blocks: 2079569
Blocks: 2066002
Pushed by aosmond@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/834a69420de8 https://hg.mozilla.org/integration/autoland/rev/b2db439c86b6 Part 1. Report 10-bit and maximum frame size decoding support from MediaCodec. r=geckoview-reviewers,gfx-reviewers,media-playback-reviewers,bradwerth,padenot,m_kato https://github.com/mozilla-firefox/firefox/commit/e4b134c9d4c9 https://hg.mozilla.org/integration/autoland/rev/873f7360b2c1 Part 2. Add MCSInfo helpers to filter MediaCodec decoding support by color depth and frame size. r=media-playback-reviewers,padenot https://github.com/mozilla-firefox/firefox/commit/7a8fd35d43cf https://hg.mozilla.org/integration/autoland/rev/9bd172c3e05a Part 3. Account for color depth and frame size when reporting MediaCodec decoding support. r=media-playback-reviewers,padenot https://github.com/mozilla-firefox/firefox/commit/bd1d773b3237 https://hg.mozilla.org/integration/autoland/rev/05f03bc6735a Part 4. Remove unused HardwareCodecCapabilityUtils code. r=geckoview-reviewers,m_kato https://github.com/mozilla-firefox/firefox/commit/0189d9804276 https://hg.mozilla.org/integration/autoland/rev/5f8d8f74f76a Part 5. Update the Android media-capabilities WPT expectations for hardware decoding. r=media-playback-reviewers,alwu
QA Whiteboard: [qa-triage-done-c161/b160]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: