Open Bug 2083164 Opened 1 month ago

Use Java/NDK MediaCodec size and frame rate capabilities when reporting Android decoder support

Categories

(Core :: Audio/Video, enhancement)

All
Android
enhancement

Tracking

()

People

(Reporter: aosmond, Unassigned)

References

(Blocks 1 open bug)

Details

Bug 2082956 limits the hardware decoding support reported on Android by frame size, using values gathered once at startup from HardwareCodecCapabilityUtils.getSupportedMimeTypesWithAccelInfo and stored in the Platform*HwDecodeMaxSize gfxVars. Those values are the per-axis upper bounds from VideoCapabilities.getSupportedWidths() and getSupportedHeights(), taken across all hardware decoders for a codec, so they are only an upper bound:

  • the decoder area limit is not captured, so a size within both maximums can still be beyond what any decoder supports (for example 4096x4096 H264 on the emulator, which reports 4096 for both axes);
  • the width and height may come from different decoders;
  • frame rate is not considered at all, so a supported size at an unsustainable frame rate is still reported as hardware decodable;
  • alignment and aspect ratio limits are ignored.

This affects navigator.mediaCapabilities.decodingInfo() (powerEfficient), and which decoding support the decoder modules report.

Exact answers need the platform to evaluate the actual width, height and frame rate:

  • Java decoding path (AndroidDecoderModule): use MediaCodecInfo.VideoCapabilities.areSizeAndRateSupported(width, height, frameRate) (or isSizeSupported when the frame rate is unknown) for the requested configuration, available on all supported Android versions.
  • NDK decoding path (FFmpegDecoderModule): on API 36+, use AMediaCodecStore_findNextDecoderForFormat, AMediaCodecInfo_getVideoCapabilities and ACodecVideoCapabilities_areSizeAndRateSupported (or AMediaCodecInfo_isFormatSupported) from libmediandk. Our minimum SDK is lower, so these need a runtime availability check, and older devices must keep falling back to the values cached at startup in gfxVars.
  • The content process answers for the NDK path through RemoteDecoderModule, from the codec support reported by the RDD, so the exact check needs to happen somewhere that can see the requested size and frame rate: either the same NDK query in the content process, if libmediandk is usable there, or in the process that hosts the decoder.
You need to log in before you can comment on or make changes to this bug.