1426725 - H.264-encoded 360 videos on Facebook crash on Mozilla Windows due to exceeding height and width limits

Status: RESOLVED DUPLICATE of bug 1422239

(Core :: Audio/Video: Playback, defect, P2)

Description

[klu.mozilla]

SUMMARY: 
The impact on Facebook's 360 videos playback is that 4k H.264-encoded 360 videos crash when played on Firefox Windows. 

ERROR:
"class mozilla::MediaResult __cdecl mozilla::WMFVideoMFTManager::ValidateVideoInfo(void): Can't decode H.264 stream because its resolution is out of the maximum limitation" 

ANALYSIS:
This error is thrown in lines 565-580 here (https://dxr.mozilla.org/mozilla-central/source/dom/media/platforms/wmf/WMFVideoMFTManager.cpp). Here is the changeset adding that adds this check (https://hg.mozilla.org/mozilla-central/rev/f67f016f5688). For 4k capable systems, the max width set here is 4096 and the max height set is 2304. However, Facebook's 4k 360 videos often exceed these limits. As an example, for https://www.facebook.com/360vidz/videos/740748536132429/, the representation dimensions reported by the browser are 2080 (width) x 3120 (height). 

RECOMMENDATION:
These 4k height and width limitations seem somewhat arbitrary; the short-term solution would be to raise the limits. However, in the future we may support even higher quality videos, so we are concerned that this issue may come back around if we simply set a higher limit. Is there a way to solve this issue in a more general case? 

Thanks and feel free to let us know if you have questions or need further examples to work with.

Comment 1

[Jean-Yves Avenard [:jya]]

The limitations aren't arbitrary, they are set by Microsoft.
https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815(v=vs.85).aspx

We have no choice but to use Windows system decoders.

Note that on Windows 7, with some version of the decoder, the maximum is 1080p

Comment 2

[Jean-Yves Avenard [:jya]]

I guess we could assume that if it can do 4096 × 2304 , then it could also do 2304 x 4096

to be confirmed by Microsoft

Comment 3

[theod]

@jya:

Excerpt from the documentation you mentioned above:

> 4096 × 2304 pixels
> 
> The maximum guaranteed resolution for DXVA acceleration is 1920 × 1088 pixels; at higher resolutions, decoding is done with DXVA, if it is supported by the underlying hardware, otherwise, decoding is done with software.

To me, this doesn't seem to say that this resolution is a hard limit, but rather that whatever the hardware limit is should be the limit. I am not familiar with the Windows APIs that you would have access to in order to assess hardware decoding capabilities, but from a pure user experience point of view, I would suggest that software decoding, if even if terribly slow, could still be a better experience than the video crashing altogether. At least the user would get some sort of playback, rather than nothing at all.

A decent amount of hardware (GPUs) are also able to decode more than 4k these days, YouTube definitely has an 8k quality for targeted towards these high-end devices for instance.

Would there be any path that could be explored in order to simply try to play the video with whatever is available? ie. hardware first and then software if hardware isn't available.

Comment 4

[Jean-Yves Avenard [:jya]]

That is not what the documentation states.
Up to 1088 pixels high, and you're guaranteed to have a hardware accelerated decoder, more than that, and you *may* get a hardware decoder if the GPU supports it, or it will fall-back to software. The maximum resolution supported being 4096*2034.

YouTube 8k streams are only using the VP9 codec, so the comparison is irrelevant. We can't decode 8k h264 with the system H264 decoders in all version of windows; we either get a not supported resolution error, or it crash hard (search Bugzilla for all the msmpeg2vdec.dll related crash)

Also, the video doesnt "crash", it emits an error event, the Video element error attribute will contain the error code. There's also a MediaError.message attribute with extended details about why the video failed to decode.

It should be up to your player to listen to those errors, and automatically fall-back to a different resolution.
We have no control over the video being serve:support d, but you do, and the User-Agent is giving you all the information required to determine that the particular video can't play.

Hope this helps.

Comment 5

[klu.mozilla]

@jya: 

Thanks for the follow-up comments; I appreciate the clarification. It would be really helpful to us if you could look into that initial fix you suggested above (if Windows can play 4096 x 2304, it should also be able to play 2304 x 4096). This would allow us to move forward with playing high quality H.264 360 videos on Firefox Windows in the short term. We can also look into improving the fallback behavior on our side.

Comment 6

[Blake Wu [:bwu][:blakewu]]

Kaku,
Is this bug related to Bug 1422239?

Comment 7

[Tzuhao Kuo [:kaku]]

(In reply to Blake Wu [:bwu][:blakewu] from comment #6)
> Kaku,
> Is this bug related to Bug 1422239?

Yes, they are.