1422239 - Some videos don't play

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[microlap]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0
Build ID: 20171128222554

Steps to reproduce:

Open  up this video https://i.imgur.com/hn0hyxb.gifv 


Actual results:

It doesn't play. Only shows the first frame and that's all. 


Expected results:

Should've played. It works as a gif, but won't work as a gifv (mp4). It works in other browsers. Can't check if it works in 56. Same happens with other videos on imgur and facebook. 

Media Tools don't work on the page for some reason (don't show anything, not even 'mediaelemets=0', nothing), so I can't show you its log

Comment 1

[YF (Yang)]

Works for me in Fx57 and Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0 ID:20171130101246.

Comment 2

[Matthias Versen [:Matti]]

Works for me with Firefox57 on Windows7

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

It is probably a perf issue. Can you attach the information in about:support ?

Comment 4

[microlap]

Attachment: information from about:support

Comment 5

[microlap]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #3)
> It is probably a perf issue. Can you attach the information in about:support
> ?

added

Comment 6

[microlap]

I should probably add I tried private windows and rebooting without addons. I know at least one person with the same issue. 

Windows 7

Comment 7

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Try setting gfx.webrender.enabled=false

Comment 8

[microlap]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #7)
> Try setting gfx.webrender.enabled=false

Doesn't work. I changed it from default (false) to try and fix this.

Comment 9

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Can you try setting media.hardware-video-decoding.enabled=false ?

Comment 10

[JW Wang [:jwwang] [:jw_wang]]

Can you:
1. go to the Windows command prompt.
2. set MOZ_DISABLE_CONTENT_SANDBOX=1
3. launch firefox.exe (usually under "C:\Program Files\Nightly\firefox.exe")
4. open about:networking#logging
5. enter "MediaDecoder:4" into the text box and click "Set Log Modules".
6. press the button "Start Logging".
7. repro the issue.
8. Press the button "Stop Logging".
9. Upload the generated log files under "C:\Users\<user>\AppData\Local\Temp\".

Comment 11

[microlap]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #9)
> Can you try setting media.hardware-video-decoding.enabled=false ?

Doesn't work

Comment 12

[microlap]

(In reply to JW Wang [:jwwang] [:jw_wang] from comment #10)
> Can you:
> 1. go to the Windows command prompt.
> 2. set MOZ_DISABLE_CONTENT_SANDBOX=1
> 3. launch firefox.exe (usually under "C:\Program Files\Nightly\firefox.exe")
> 4. open about:networking#logging
> 5. enter "MediaDecoder:4" into the text box and click "Set Log Modules".
> 6. press the button "Start Logging".
> 7. repro the issue.
> 8. Press the button "Stop Logging".
> 9. Upload the generated log files under
> "C:\Users\<user>\AppData\Local\Temp\".

Hello Sorry But you'll have to explain this to me. 
2. Just like that in cmd? Cmd doesn't respond with anything, it doesn't show any  errors, too, so I'm not sure if I did this right
9. The log file is generated, but it's empty. Even if I open some 'working' media, so I must've done something wrong. 

I did all the steps from 3 to 9 exactly as you said, but I have some doubts about the '2'.

Comment 13

[JW Wang [:jwwang] [:jw_wang]]

You will get empty log files if '2' is not done correctly.

Type "set MOZ_DISABLE_CONTENT_SANDBOX=1" and press the 'enter' key.
Type "set" and press the 'enter' key to list all environment variables and check if MOZ_DISABLE_CONTENT_SANDBOX is also listed.

Comment 14

[JW Wang [:jwwang] [:jw_wang]]

Btw, you need to close all firefox instances before step 1.

Comment 15

[microlap]

Okay, so apparently the logfile name stated in the logging page is empty, but the 'child' one with a different number next to it works. At least I hope it works. 

I'm adding the generated file to the main post. Steps I did:
1. Open https://i.imgur.com/hn0hyxb.gifv - doesn't play
2. Open https://i.imgur.com/hn0hyxb - doesn't play

Thank you

Comment 16

[microlap]

Attachment: mediadecoder log

Comment 17

[JW Wang [:jwwang] [:jw_wang]]

Thanks for the logs.

Can you replace "MediaDecoder:4" with "MediaDecoder:4,nsMediaElement:4,nsMediaElementEvents:4" in step 5 and try again? Thanks!

Btw, please open one link only.

Comment 18

[microlap]

Attachment: log.txt-child.5964

Comment 19

[microlap]

(In reply to JW Wang [:jwwang] [:jw_wang] from comment #17)
> Btw, please open one link only.

Done.

Comment 20

[JW Wang [:jwwang] [:jw_wang]]

Thanks for the logs.

Can you then go to Tools -> Web Developer -> Web console and see if there any interesting error messages?

Comment 21

[microlap]

(In reply to JW Wang [:jwwang] [:jw_wang] from comment #20)
> Thanks for the logs.
> 
> Can you then go to Tools -> Web Developer -> Web console and see if there
> any interesting error messages?

Actually, yes!

I think this one is pretty interesting:
Error Code: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005)
Details: class mozilla::MediaResult __cdecl mozilla::WMFVideoMFTManager::ValidateVideoInfo(void): Can't decode H.264 stream because its resolution is out of the maximum limitation

So I guess there's an upper limit to the h264 videos resolution? That's a weird one. 
Thanks, now I have at least *something* about this issue.

Comment 22

[JW Wang [:jwwang] [:jw_wang]]

Hi Jya,
Do you have any idea about this error? How can we fix it? Don't we fallback to software decoders when hardware ones fail to init?

Comment 23

[Jean-Yves Avenard [:jya]]

We have a fall-back in place only if the decoder failed to initialize.we always assume that if it got initialized properly, then decoding will work.

This could be a recent regression though, IIRC testing the résolution was done during the initialization.

Comment 24

[Blake Wu [:bwu][:blakewu]]

John,
Are you available to check this bug?

Comment 25

[John Lin [:jhlin][:jolin]]

Sorry for the late reply. I won't be in office for quite some time and don't have a windows machine to check the problem.

Comment 26

[John Lin [:jhlin][:jolin]]

Looks like the machine fails 'Is4KCapable' test at [1] and it's a 720x1272 video which exceeds the max height value (1088), as :microlap have found in comment 21.


[1] https://searchfox.org/mozilla-central/source/dom/media/platforms/wmf/WMFVideoMFTManager.cpp#570

Comment 27

[microlap]

sorry to chim in but if it helps I can find more videos that don't work

Comment 28

[Tzuhao Kuo [:kaku]]

(In reply to microlap from comment #27)
> sorry to chim in but if it helps I can find more videos that don't work

Hi, microlap, 

Thanks for reporting this issue and providing useful debug information.
More videos would be appreciated as long as it doesn't overwhelm you.

Comment 29

[Tzuhao Kuo [:kaku]]

(In reply to John Lin [:jolin][:jhlin] from comment #26)
> Looks like the machine fails 'Is4KCapable' test at [1] and it's a 720x1272
> video which exceeds the max height value (1088), as :microlap have found in
> comment 21.
> 
> 
> [1]
> https://searchfox.org/mozilla-central/source/dom/media/platforms/wmf/
> WMFVideoMFTManager.cpp#570

I think this is one of the problems.
The spec (https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815(v=vs.85).aspx) says that the resolution limitation is 4096 × 2304 pixels but not constraints it to be width <= 4096 and height <= 2304. I think it could be any configuration as long as the total pixels are less than 4096 × 2304.

In Windows 7 case, the limitation is 1920 x 1080, which I think is @microlap's case.

Comment 30

[microlap]

They're quite easy to find, so no problem. Please keep in mind I don't what is exactly *in* the videos so I'm sorry if there is something inappropriate. The problem is the same — I only see the first frame (unless I force it to play as a gif)

https://i.imgur.com/pwuwMay.gifv

https://i.imgur.com/L6H75cu.gifv

https://i.imgur.com/3njIi6R.gifv

https://i.imgur.com/5YCZnL3.gifv

If I understand your comment correctly this is a h.264 + Win7 issue, not firefox', right? Then I guess there's nothing to be done...

Comment 31

[Tzuhao Kuo [:kaku]]

(In reply to microlap from comment #30)
> They're quite easy to find, so no problem. Please keep in mind I don't what
> is exactly *in* the videos so I'm sorry if there is something inappropriate.
> The problem is the same — I only see the first frame (unless I force it to
> play as a gif)
> 
> https://i.imgur.com/pwuwMay.gifv
> 
> https://i.imgur.com/L6H75cu.gifv
> 
> https://i.imgur.com/3njIi6R.gifv
> 
> https://i.imgur.com/5YCZnL3.gifv

Thanks a lot! All these videos match my guess, they all have height > 1080 pixels.

> 
> If I understand your comment correctly this is a h.264 + Win7 issue, not
> firefox', right? Then I guess there's nothing to be done...

No, it's Firefox's issue.
We misunderstood the dimension limitation to be "width <= 1920 AND height <= 1080", for Windows 7 without 4K supporting, which should be your computer's configuration.

However, the real limitation specified by Microsoft is looser. It accepts andy width and height combination as long as the total pixel count is under 1920x1080 pixels.

A patch is following soon.

Comment 32

[Tzuhao Kuo [:kaku]]

Attachment: bug 1422239 - relax the resolution limitation of WMF H264 decoder;

Review commit: https://reviewboard.mozilla.org/r/210888/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/210888/

Comment 33

[Tzuhao Kuo [:kaku]]

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7b84a28cc8efc1a04188031a567f239cf24e5332

Comment 34

[Jean-Yves Avenard [:jya]]

Comment on attachment 8940640 [details]
bug 1422239 - relax the resolution limitation of WMF H264 decoder;

https://reviewboard.mozilla.org/r/210888/#review216582

::: dom/media/platforms/wmf/WMFVideoMFTManager.cpp:576
(Diff revision 1)
>    static const int32_t MIN_H264_FRAME_DIMENSION = 48;
>    static const int32_t MAX_H264_FRAME_WIDTH = Is4KCapable ? 4096 : 1920;
>    static const int32_t MAX_H264_FRAME_HEIGHT = Is4KCapable ? 2304 : 1088;
>  
> -  if (mVideoInfo.mImage.width > MAX_H264_FRAME_WIDTH  ||
> -      mVideoInfo.mImage.height > MAX_H264_FRAME_HEIGHT) {
> +  // Test both landscape and portrait configurations.
> +  if ((mVideoInfo.mImage.width > MAX_H264_FRAME_WIDTH  ||

if it's the pixel count you want to test again, then please use that, using CheckedInt.

I still think that we have no guarantee that this "understanding" of MS documentation is more correct than the previous one. It's just a guess, which happens to work here.

The proper method would be to let it decode regardless of the dimensions and have a fallback from hardware to software if things failed.

right now, we have none. So if we create a HW decoder and initialization succeeded, future decoding must work otherwise we'll get a fatal error.

Comment 35

[Tzuhao Kuo [:kaku]]

(In reply to Jean-Yves Avenard [:jya] from comment #34)
> Comment on attachment 8940640 [details]
> bug 1422239 - relax the resolution limitation of WMF H264 decoder;
> 
> https://reviewboard.mozilla.org/r/210888/#review216582
> 
> ::: dom/media/platforms/wmf/WMFVideoMFTManager.cpp:576
> (Diff revision 1)
> >    static const int32_t MIN_H264_FRAME_DIMENSION = 48;
> >    static const int32_t MAX_H264_FRAME_WIDTH = Is4KCapable ? 4096 : 1920;
> >    static const int32_t MAX_H264_FRAME_HEIGHT = Is4KCapable ? 2304 : 1088;
> >  
> > -  if (mVideoInfo.mImage.width > MAX_H264_FRAME_WIDTH  ||
> > -      mVideoInfo.mImage.height > MAX_H264_FRAME_HEIGHT) {
> > +  // Test both landscape and portrait configurations.
> > +  if ((mVideoInfo.mImage.width > MAX_H264_FRAME_WIDTH  ||
> 
> if it's the pixel count you want to test again, then please use that, using
> CheckedInt.
I had tried, but the "pixel-count-guess" does not always work. 
I created a video with dimension 8192 x 1152, which doubles the max_with(4096) and halfs the max_height(2304).
Use it to initialize a WFM decoder fails at 
https://searchfox.org/mozilla-central/rev/03877052c151a8f062eea177f684a2743cd7b1d5/dom/media/platforms/wmf/MFTDecoder.cpp#95
, which returns MF_E_INVALIDMEDIATYPE error code.

> I still think that we have no guarantee that this "understanding" of MS
> documentation is more correct than the previous one. It's just a guess,
> which happens to work here.
Yes, so I think attachment 8940640 [details] is an acceptable assumption for relaxing the dimension limitation.
The idea is, if the video is in landscape mode, we constraint it to be width <= 4096 and height <= 2304.
For portrait mode, it should be width <= 2304 and height <= 4096.
If I understand you correctly, I think your comment (bug 1426725 comment 2) also has the same idea!?

> The proper method would be to let it decode regardless of the dimensions and
> have a fallback from hardware to software if things failed.
I am afraid that it doesn't help because we are not able to initialize WMF HW and SW decoder if we don't relax the resolution limitation. And, we don't use ffmpeg to decode H264 on Windows, so WMF is the only possibility.

> right now, we have none. So if we create a HW decoder and initialization
> succeeded, future decoding must work otherwise we'll get a fatal error.
Yes, we only dynamically fallback to SW decoder while GPU process crashes.

Comment 36

[Tzuhao Kuo [:kaku]]

(In reply to Tzuhao Kuo [:kaku] from comment #35)
> > I still think that we have no guarantee that this "understanding" of MS
> > documentation is more correct than the previous one. It's just a guess,
> > which happens to work here.
> Yes, so I think attachment 8940640 [details] is an acceptable assumption for
> relaxing the dimension limitation.
> The idea is, if the video is in landscape mode, we constraint it to be width
> <= 4096 and height <= 2304.
> For portrait mode, it should be width <= 2304 and height <= 4096.
> If I understand you correctly, I think your comment (bug 1426725 comment 2)
> also has the same idea!?
@jya,
I think we can go with this approach, at least it doesn't make things worse.
The original accepted cases are not impacted.
The newly accepted cases might be decoded successfully; If they're not, we don't lose anything.
WDYT?

Comment 37

[Jean-Yves Avenard [:jya]]

If we want to relax the rules, let's relax them. Using the pixel count as limit sounds like an acceptable compromise (no idea on how well the decoder will behave with a 10 x 16536 videos though)

(In reply to Tzuhao Kuo [:kaku] from comment #35)
> > if it's the pixel count you want to test again, then please use that, using
> > CheckedInt.
> I had tried, but the "pixel-count-guess" does not always work. 
> I created a video with dimension 8192 x 1152, which doubles the
> max_with(4096) and halfs the max_height(2304).
> Use it to initialize a WFM decoder fails at 
> https://searchfox.org/mozilla-central/rev/
> 03877052c151a8f062eea177f684a2743cd7b1d5/dom/media/platforms/wmf/MFTDecoder.
> cpp#95
> , which returns MF_E_INVALIDMEDIATYPE error code.
> 

why does it matter if initialisation fail later?
the outcome for the user is still the same:
fatal decoding error.

That we fail creating the decoder or initialising it is not of importance.
The original limit was put in place because we had hard crashes happening sometimes. But those cases no longer matter as we have the GPU process crash fallback available.


> > I still think that we have no guarantee that this "understanding" of MS
> > documentation is more correct than the previous one. It's just a guess,
> > which happens to work here.
> Yes, so I think attachment 8940640 [details] is an acceptable assumption for
> relaxing the dimension limitation.
> The idea is, if the video is in landscape mode, we constraint it to be width
> <= 4096 and height <= 2304.
> For portrait mode, it should be width <= 2304 and height <= 4096.
> If I understand you correctly, I think your comment (bug 1426725 comment 2)
> also has the same idea!?

To me, relaxing the rules by adding a different set doesn't make much sense. Either we fully relax, or we don't
Rules that are based on a likely wrong assumptions on what can and can't be done are bad.

So if we are to choose, I prefer a pixel count limitation, and let the decoder initialization fail later.

> 
> > The proper method would be to let it decode regardless of the dimensions and
> > have a fallback from hardware to software if things failed.
> I am afraid that it doesn't help because we are not able to initialize WMF
> HW and SW decoder if we don't relax the resolution limitation. And, we don't
> use ffmpeg to decode H264 on Windows, so WMF is the only possibility.

obviously this assume that this check has first been removed.

> 
> > right now, we have none. So if we create a HW decoder and initialization
> > succeeded, future decoding must work otherwise we'll get a fatal error.
> Yes, we only dynamically fallback to SW decoder while GPU process crashes.

and that is the proper fix to this bug, and it needs to be done at some stage.

Comment 38

[Tzuhao Kuo [:kaku]]

(In reply to Jean-Yves Avenard [:jya] from comment #37)

> To me, relaxing the rules by adding a different set doesn't make much sense.
> Either we fully relax, or we don't
> Rules that are based on a likely wrong assumptions on what can and can't be
> done are bad.
> 
> So if we are to choose, I prefer a pixel count limitation, and let the
> decoder initialization fail later.
Okay, let's go with the pixel count limitation.

> > 
> > > right now, we have none. So if we create a HW decoder and initialization
> > > succeeded, future decoding must work otherwise we'll get a fatal error.
> > Yes, we only dynamically fallback to SW decoder while GPU process crashes.
> 
> and that is the proper fix to this bug, and it needs to be done at some
> stage.
Hmm......., let me open a new bug for it.

Comment 39

[Tzuhao Kuo [:kaku]]

Comment on attachment 8940640 [details]
bug 1422239 - relax the resolution limitation of WMF H264 decoder;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/210888/diff/1-2/

Comment 40

[Jean-Yves Avenard [:jya]]

Comment on attachment 8940640 [details]
bug 1422239 - relax the resolution limitation of WMF H264 decoder;

https://reviewboard.mozilla.org/r/210888/#review217460

Comment 41

[Tzuhao Kuo [:kaku]]

Try looks good: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ed55a3bdae1266dae809025b68aff77a960d6169
Thanks for the review.

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cede5634c11b
relax the resolution limitation of WMF H264 decoder; r=jya

Comment 43

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/cede5634c11b