1764425 - Videos with non-1:1 pixel aspect ratios do not correctly scale.

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Zaggy1024]

Attachment: continuous_vp9_res.webm

Currently, VP9 and AV1 videos with the render size not equal to the frame size are not scaled correctly. An example of this is this attachment from Bug 1697316. Another example will be attached.

The pixel aspect ratio should be derived from the frame and render sizes and stored by MediaChangeMonitor to be able to apply the ratio to any subsequent image sizes found in-band.

Note that VP9 allows a sample to change the render size independently of the frame size, so in cases where that flag is set, we should probably also change the pixel aspect ratio to match. I currently don't have any test cases for that, as I haven't found a way to make libvpx write a render size to the bitstream with FFmpeg. If there is a way, it would be good to test that in other media players for consistency.

Comment 1

[Zaggy1024]

(In reply to Zaggy1024 from comment #0)

Currently, VP9 and AV1 videos with the render size not equal to the frame size are not scaled correctly. An example of this is this attachment from Bug 1697316. Another example will be attached.

In this attachment, the video is displayed at almost double the size that is intended, because MediaChangeMonitor assumes that it should leave the render resolution unchanged since the aspect ratio doesn't match. Instead, the render size after it encounters the first sample should have the height set to 384px, and the width should be scaled by the PAR to 681px.

In continuous_vp9_res.webm, the render resolution update is rejected on the third and last sequence in the video, as it is an imperfect 16:9 resolution, 1366x768 with a ratio of 16.0078125:9. When scaling by the pixel aspect ratio instead, the resolution is correctly set to 1366x768.

Comment 2

[Zaggy1024]

Attachment: Bug 1764425 - Scale videos based on the pixel aspect ratio from containers or VP9 keyframe headers that specify render size. r=alwu

Fixes issues where frame size changes in-band would not be applied if the aspect ratio changed.

Depends on D143558

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f2419b3e8f7c
Scale videos based on the pixel aspect ratio from containers or VP9 keyframe headers that specify render size. r=alwu

Comment 4

[Norisz Fay [:noriszfay]]

Backed out for causing mochitest failures on test_eme_waitingforkey.html

Backout link

Push with failures

Failure log

Failure line(s): TEST-UNEXPECTED-FAIL | dom/media/test/test_eme_waitingforkey.html | Test timed out!

Comment 5

[Zaggy1024]

Fix is attached to Bug 1764478.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6913a3898617
Scale videos based on the pixel aspect ratio from containers or VP9 keyframe headers that specify render size. r=alwu

Comment 7

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/6913a3898617