1917874 - HEVC Content on twitch does not have the correct width or height

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[John Bowers]

Attachment: Screenshot 2024-09-10 at 9.39.34 AM.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36

Steps to reproduce:

Searched for HEVC videos on twitch. Usually there are one or two running. Twitch is running an HEVC beta currently, and there are a limited number of users streaming in HEVC. Make sure to enable HEVC in about:config, otherwise you won't see the HEVC quality.

Actual results:

Any video I find has slightly larger than expected dimensions. Usually height.

Expected results:

I am a twitch developer. This issue is related to the HEVC conformance window. I had to fix this exact issue once already. We are encoding mostly with nvidia cards, and nvidia sets the conformance window flag in the HEVC SPS. If it does, the width and height calculations are slightly more complicated than just taking the pic_width_in_luma_samples and pic_height_in_luma_samples. For a 1080 stream this is often 1088, so I know this is exactly what you are doing (and it makes sense, I did it myself). See the attached screenshot. Refer to the HEVC spec for how to calculate the actual width and height when the conformance window flag is set, here are the relevant bits "pic_width_in_luma_samples specifies the width of each decoded picture in units of luma samples.
pic_width_in_luma_samples shall not be equal to 0 and shall be an integer multiple of MinCbSizeY.
pic_height_in_luma_samples specifies the height of each decoded picture in units of luma samples.
pic_height_in_luma_samples shall not be equal to 0 and shall be an integer multiple of MinCbSizeY.
conformance_window_flag equal to 1 indicates that the conformance cropping window offset parameters follow next in
the SPS. conformance_window_flag equal to 0 indicates that the conformance cropping window offset parameters are not
present.
conf_win_left_offset, conf_win_right_offset, conf_win_top_offset and conf_win_bottom_offset specify the samples of
the pictures in the CVS that are output from the decoding process, in terms of a rectangular region specified in picture
coordinates for output. When conformance_window_flag is equal to 0, the values of conf_win_left_offset,
conf_win_right_offset, conf_win_top_offset and conf_win_bottom_offset are inferred to be equal to 0.
The conformance cropping window contains the luma samples with horizontal picture coordinates from
SubWidthC * conf_win_left_offset to pic_width_in_luma_samples − ( SubWidthC * conf_win_right_offset + 1 ) and
vertical picture coordinates from SubHeightC * conf_win_top_offset to
pic_height_in_luma_samples − ( SubHeightC * conf_win_bottom_offset + 1 ), inclusive.
The value of SubWidthC * ( conf_win_left_offset + conf_win_right_offset ) shall be less than
pic_width_in_luma_samples, and the value of SubHeightC * ( conf_win_top_offset + conf_win_bottom_offset ) shall be
less than pic_height_in_luma_samples."

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Cocoa' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[John Bowers]

I am sure widget: Cocoa must be an incorrect classification for this bug.

Comment 3

[John Bowers]

For finding HEVC videos on twitch keep in mind that the beta cohort isn't huge right now, so you might not always be able to find a live HEVC stream, but if you scroll down to past broadcasts you should be able to easily find some recordings that have been tagged as HEVC. Only resolutions above 720p will be in HEVC.

Comment 4

[John Bowers]

This is still happening for HEVC streams, we aren't creating 1080p HEVC renditions anymore, but it happens with any 4k renditions which show up in the video stats as 2176x3840 instead of 2160x3840. Just open a twitch 4k stream (find one here that is 3840x2160 - https://eb.rodney.io/) in firefox and click on the settings gear in the bottom right corner -> advanced -> show video stats, You will see 2176x3840. If you do the same thing in chrome you will see 2160x3840. Additionally there is black boundary around the video that shouldn't be there, which seems to be because of the incorrect dimensions. If you properly handle the conformance window flag, this should go away.

Comment 5

[Alastor Wu [:alwu]]

Thank for reminding, I should be able to work on this next week if there is no new high priority tasks or interrupt. Set NI on myself.

Comment 6

[Alastor Wu [:alwu]]

Attachment: Bug 1917874 - part1 : apply the conformance cropping window on the decoded image.

Comment 7

[Alastor Wu [:alwu]]

Attachment: Bug 1917874 - part2 : add a test case.

Depends on D242616

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d4be0509ba13
part1 : apply the conformance cropping window on the decoded image. r=media-playback-reviewers,padenot
https://hg.mozilla.org/integration/autoland/rev/6921fdd63b36
part2 : add a test case. r=media-playback-reviewers,padenot

Comment 9

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/d4be0509ba13
https://hg.mozilla.org/mozilla-central/rev/6921fdd63b36