1422681 - Unable to set pixel aspect ratio of video track in MP4 container played via MSE

Status: RESOLVED WONTFIX

(Core :: Audio/Video: Playback, defect)

Description

[goncharov]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36

Steps to reproduce:

1. Create video element, media source and add MP4 AVC source buffer.
2. Feed source buffer with init segment (ftyp + moov), containing "tkhd" box with relevant width and height parameters and "pasp" box with relevant hSpacing and vSpacing parameters.
3. Feed source buffer with fragmented MP4 stream.

974297


Actual results:

The video is displayed with initial aspect ratio regardless of width, height, hSpacing and vSpacing parameters.


Expected results:

The video should be displayed with predefined aspect ratio.
Firefox behaves correctly with WEBM container and VP8/9 codec, setting desired aspect ratio via DisplayWidth and DisplayHeight parameters. Given scenario works correctly in Chrome and Safari browsers.

Comment 1

[goncharov]

There was similar issue https://bugzilla.mozilla.org/show_bug.cgi?id=974297 that was fixed 2 years ago, but looks like it's not relevant anymore for MP4 container.

Comment 2

[Jean-Yves Avenard [:jya]]

(In reply to goncharov from comment #0)
> User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Chrome/62.0.3202.94 Safari/537.36
> 
> Steps to reproduce:
> 
> 1. Create video element, media source and add MP4 AVC source buffer.
> 2. Feed source buffer with init segment (ftyp + moov), containing "tkhd" box
> with relevant width and height parameters and "pasp" box with relevant
> hSpacing and vSpacing parameters.
> 3. Feed source buffer with fragmented MP4 stream.
> 

> 974297
> 
> 
> Actual results:
> 
> The video is displayed with initial aspect ratio regardless of width,
> height, hSpacing and vSpacing parameters.

aspect ratio in a h264 stream is defined in the SPS NAL.

Not the container.

Comment 3

[goncharov]

That's correct, video frame aspect ratio is defined in the SPS. But you can change it by e.g. setting pixel aspect ratio in container. That's known usecase and that's what the issue is about.

Comment 4

[goncharov]

The issue fixed 3 years ago is named "Media Recording - support aspect-ratio change in ISO media format container". Maybe it can give you more understanding of what I'm trying to say.

Comment 5

[Jean-Yves Avenard [:jya]]

that may well be, but via MSE (as well as any mp4/h264 playback), the aspect ratio is read in the bytestream of the H264 data.

this is how all major video sites pass the information, and we ignore purposefully what is set in the container.

More often than not, the aspect ratio in the container / init segment is actually incorrect. So ignoring it and using the H264 SPS NAL was done on purpose.

This is specially true with AVC3 streams like used by the BBC. They use a common init segment, describing their base video streams (resolution/aspect ratio). the bytestream will indicate what actual resolutions/aspect ratio to use as it is encountered.

We also have no control about what some H264 decoders are doing. They too use the bytestream to determine the size output by the decoder.

The container display information will be ignored, not doing so would break the playback of many content providers.

Comment 6

[goncharov]

ok, I understand, thanks for the explanation.