1154162 - Gonk H264 decoder doesn't properly handle change of resolution: incorrect aspect ratio

Status: RESOLVED DUPLICATE of bug 1243538

(Core :: Audio/Video: Playback, defect)

Description

[Jean-Yves Avenard [:jya]]

As MSE isn't activated on gonk yet, the issue hasn't been a problem. But it will be once MSE is activated.

When using MSE, the Gonk H264 decoder is encapsulated in a SharedDecoderManager which will re-use the current H264 decoder for all content. That means that the MediaRawData (aka MP4Sample) that is passed to GonkDecoderManager::Input can be of different resolution and size.

The GonkVideoDecoderManager will handle change of video resolution and will call GonkVideoDecoderManager::SetVideoFormat, which will retrieve the new width and height found in the H264 SPS NAS.

It will then compare this height and width with the one original provided in the VideoInfo originally provided to the GokVideoDecoderManager.

It will then resize that display window according to the original aspect ratio. This is the wrong behaviour. The aspect ratio can't be reused as it's only valid for the original resolution provided at the time the decoder was created..

We have implemented a mechanism to inform the h264 decoder that a format/resolution is about to happen, and that the next MediaRawData passed to the decoder will be of a different resolution.

  // ConfigurationChanged will be called to inform the video or audio decoder
  // that the format of the next input sample is about to change.
  // If video decoder, aConfig will be a VideoInfo object.
  // If audio decoder, aConfig will be a AudioInfo object.
  virtual nsresult ConfigurationChanged(const TrackInfo& aConfig)

The GonkVideoDecoder should implement that API, and monitor change of resolution there and retrieve the new correct aspect ratio.

Looking at android source code, they only appear to provide the decoded image width/height, not its display size unfortunately.

Additionally, upon reviewing the OMX decoder, it appears that odd-size resolution aren't always correct.

Comment 1

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Jean-Yves - is this still an issue?

Comment 2

[Jean-Yves Avenard [:jya]]

I think so...
Will mostly be a problem with sites like Netflix which makes great use of aspect ratio. YouTube doesn't so the issue isn't noticeable there.

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Blake - I'll leave this for you to prioritise.

Comment 4

[Blake Wu [:bwu][:blakewu]]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #3)
> Blake - I'll leave this for you to prioritise.
Thanks for this info. We will check it.

Comment 5

[Jean-Yves Avenard [:jya]]

not a MSE issue ; and it doesn't appear to attract much concerns unfortunately

Comment 6

[Jean-Yves Avenard [:jya]]

while working on bug 1243538 found a rather elegant solution for it.

Comment 7

[Jean-Yves Avenard [:jya]]

Attachment: MozReview Request: Bug 1154162: P1. Properly handle aspect ratio in gonk video decoder. r?alfredo

The display size were used for the picture size which would be wrong if any aspect ratio was set. Additionally handle bug 1243538 specifically for gonk video decoder.

Review commit: https://reviewboard.mozilla.org/r/46673/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/46673/

Comment 8

[Jean-Yves Avenard [:jya]]

moved to bug 1243538 as it makes more sense to have it as a block.

once bug 1243538, this bug will be fixed.