1316571 - Video gets stuck after playing for a moment

Status: ASSIGNED

(Core :: Audio/Video: Playback, defect, P3)

Description

[6lobe]

Certain video files get stuck after playing for a short time (typically less than 1 second).

Here is one such file:

https://giant.gfycat.com/ForcefulMeanArcticfox.mp4
https://giant.gfycat.com/ForcefulMeanArcticfox.webm

And here is the regression range:

29:40.91 INFO: Oh noes, no (more) inbound revisions :(
29:40.91 INFO: Last good revision: 8146a675a477c67534886dcf10937e62d0c1da68
29:40.91 INFO: First bad revision: 4f39f2e1324b8c631dc700f86500f49b9680458d
29:40.91 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=8146a675a477c67534886dcf10937e62d0c1da68&tochange=4f39f2e1324b8c631dc700f86500f49b9680458d

29:42.21 INFO: Looks like the following bug has the changes which introduced the regression:
https://bugzilla.mozilla.org/show_bug.cgi?id=1258870

Comment 1

[Jean-Yves Avenard [:jya]]

Could you please try again with Nightly and see if you still have the problem?

Comment 2

[6lobe]

Yes, I have this problem with latest Nightly.

Comment 3

[Jean-Yves Avenard [:jya]]

any chance you could record a video showing the problem?

does it do that with all the machines you have or just one in particular?

could you post the output of about:support ?

Thank you

Comment 4

[Jean-Yves Avenard [:jya]]

:cpearce, may want to look into this... Wonder if this will be improved with yesterday's patch. Though here we have a webm, they can't be overlapping frames there normally (haven't looked into the frames time)

Comment 5

[6lobe]

I'd prefer not to go through the hassle of recording a video if possible. The video playback just stops after a playing for a short moment.

I checked that this does not happen on a powerful x64 machine.

This machine is a relatively slow laptop, I will attach info.

Comment 6

[6lobe]

Attachment: about:support

Here is the about:support.

Comment 7

[6lobe]

Attachment: output of lscpu

Here is CPU info from lscpu.

Comment 8

[Jean-Yves Avenard [:jya]]

Are you sure you've tried today's nightly? One with bug 1307546 in...

We hope that it should fix such issue.

Comment 9

[Jean-Yves Avenard [:jya]]

Actually, report is shown to be with yesterday's nightly :(

Comment 10

[Jean-Yves Avenard [:jya]]

Could you please install the following extension:
https://addons.mozilla.org/en-US/firefox/addon/about-media/

When the video has stalled, open a new tab and go to about:media and post here the output of it...

Thank you.

Comment 11

[6lobe]

Attachment: about:media output

Here is about:media output from both videos after stalling.

And yes on latest nightly (2016-11-10).

Comment 12

[Chris Pearce [:cpearce (Not reading bugmail)]]

I cannot reproduce this on my Ubuntu 16.04 x64 system. Maybe this will be fixed by bug 1316571? Or maybe it only affects older 32bit systems?

Comment 13

[Chris Pearce [:cpearce (Not reading bugmail)]]

I meant maybe this will be fixed by bug 1316543.

Comment 14

[Jean-Yves Avenard [:jya]]

(In reply to Chris Pearce (:cpearce) from comment #13)
> I meant maybe this will be fixed by bug 1316543.

If so, this will need to be uplifted asap, and hopefully available in 50.1

Comment 15

[Andrew Overholt [:overholt]]

Should uplifting the fix in bug 1316543 block shipping 50?

Comment 16

[6lobe]

The videos are still stalling so this did not get fixed by bug 1316543.

Comment 17

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to 6lobe from comment #16)
> The videos are still stalling so this did not get fixed by bug 1316543.

I doubt bug 1316543 has reached Nightly yet. Try again tomorrow.

Comment 18

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Andrew Overholt [:overholt] from comment #15)
> Should uplifting the fix in bug 1316543 block shipping 50?

We're actually not sure yet whether this fixes any issues; no one in engineering can repro the bug we've got filed.

Comment 19

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Tracked for 50+.

Comment 20

[Chris Pearce [:cpearce (Not reading bugmail)]]

6lobe: I note that these videos don't have an audio stream, does this problem only happen for you for videos which don't have an audio stream?

Comment 21

[6lobe]

Now that you mention it, I think so. However I cant be sure.

Comment 22

[6lobe]

Scratch that. This does happen with videos that have audio.

Comment 23

[Milan Sreckovic [:milan] (needinfo for best results)]

It's possible I'm seeing this problem on an old MacBook Air, on Facebook videos.

Comment 24

[Chris Pearce [:cpearce (Not reading bugmail)]]

Milan: Are you testing with 49 or 50 on your Macbook Air? And what year and OS version is it?

Comment 25

[Milan Sreckovic [:milan] (needinfo for best results)]

I believe it was 50.  2010 Air, Nvidia discrete graphics, 10.11 OS.  I've switched to Nightly though, so let me see if this still happens.

Comment 26

[Jean-Yves Avenard [:jya]]

(In reply to Milan Sreckovic [:milan] from comment #25)
> I believe it was 50.  2010 Air, Nvidia discrete graphics, 10.11 OS.  I've
> switched to Nightly though, so let me see if this still happens.

the specs of the macbook air are very similar to a mac mini 2010 I have here (2GHz C2D and nvidida 320M), problem is that it only has 10.7 installed on it which is no longer supported :(

Comment 27

[Andrew Overholt [:overholt]]

Is anyone still seeing this?

Comment 28

[6lobe]

(In reply to Andrew Overholt [:overholt] from comment #27)
> Is anyone still seeing this?

Yes, still the same issue.

Comment 29

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

I see the video linked above play, stall, buffer and then continue.

Comment 30

[6lobe]

(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) [afk 24 Dec - 8 Jan] from comment #29)
> I see the video linked above play, stall, buffer and then continue.

This is not about buffering or network performance. This is about video playback itself, Firefox is unable to play back these (and many other) video files.

For example Chromium has no trouble playing these videos, smooth playback from start to finish. But Firefox stutters and stalls.

Comment 31

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

I think we need to get to the bottom of this. There is something funny going on.

Comment 32

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Seems likely that 54 is also affected. 
I just tried this on my MacBook Air (Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0) and the behavior was very strange. The video appeared to be progressing by the time display (rather than the slider staying in the same place and appearing to be buffering), but nothing else moved on screen until about 20 seconds went by. Then the video started actually playing. On a reload, the video played normally. 
Is this something that QE could help with?

Comment 33

[Gerry Chang [:gchang]]

Hi Blake,
Can you help find an owner for this?

Comment 34

[Blake Wu [:bwu][:blakewu]]

JW,
Please check it. 
Thanks.

Comment 35

[JW Wang [:jwwang] [:jw_wang]]

Can you run the following command to repro the issue and upload the logs? Thanks!

MOZ_LOG=MediaDecoder:4 ./firefox path-to-the-video

Comment 36

[6lobe]

Attachment: MOZ_LOG=MediaDecoder:4

Here are the logs you requested.

Comment 37

[JW Wang [:jwwang] [:jw_wang]]

(In reply to 6lobe from comment #36)
> Created attachment 8848013 [details]
> MOZ_LOG=MediaDecoder:4
> 
> Here are the logs you requested.

Thanks for the quick response.

There are lots of skipping-video-decode messages in the log.
[MediaPlayback #2]: D/MediaDecoder Decoder=0x9c150500 state=DECODING Skipping video decode to the next keyframe lowAudio=0 lowVideo=1 lowUndecoded=0 async=1
[MediaPlayback #2]: D/MediaDecoder Decoder=0x9c150500 state=DECODING Skipping video decode to the next keyframe lowAudio=0 lowVideo=1 lowUndecoded=0 async=1

Apparently video decoding is too slow to keep up. Since bug 1258870, we discard late frames so they are not presented to the user. So playback seems to get stuck but in fact, video frames are decoded and discarded repeatedly until EOS is reached.

Can you try the video on Chrome or other browsers and observe if there is any frame-dropping? This will help  us confirm if slow decoding is due to insufficient machine power or inefficiency in Firefox media code.

Thanks!

Comment 38

[6lobe]

As I said a few comments up, Chromium has no trouble playing these videos. Smooth playback from start to finish.

Comment 39

[JW Wang [:jwwang] [:jw_wang]]

https://queue.taskcluster.net/v1/task/ZVjJWZSsRTaadTbRHt92Gw/runs/0/artifacts/public/build/target.tar.bz2

Can you try this build and run "MOZ_LOG=MediaDecoder:4,MediaSample:4 ./firefox path-to-the-video" again? Thanks!

Note you have to close all Firefox windows before starting the test.

Comment 40

[6lobe]

Attachment: MOZ_LOG=MediaDecoder:4,MediaSample:4 with build from comment 39

New log with new build.

Comment 41

[JW Wang [:jwwang] [:jw_wang]]

(In reply to 6lobe from comment #40)
> Created attachment 8849015 [details]
> MOZ_LOG=MediaDecoder:4,MediaSample:4 with build from comment 39
> 
> New log with new build.

[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [83416,100100] dif=77052 clock=-1
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [100100,116783] dif=29741 clock=-1
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [116783,133466] dif=33683 clock=-1
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [133466,150150] dif=45035 clock=19996
[MediaPlayback #2]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [150150,166833] dif=41269 clock=61330
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [166833,183516] dif=44747 clock=106266
[MediaPlayback #2]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [183516,200200] dif=37866 clock=144225
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [200200,216883] dif=38038 clock=182387
[MediaPlayback #2]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [216883,233566] dif=25597 clock=208085
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [233566,250250] dif=43833 clock=252054
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [250250,266933] dif=40293 clock=292426
[MediaPlayback #2]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [266933,283616] dif=52461 clock=344958
[MediaPlayback #1]: D/MediaSample Decoder=0x7f4d2e4756f0 OnVideoDecoded [283616,300300] dif=26182 clock=371206

|dif| is the decoding time for each video frame observed by MDSM. Video decoding is too slow (each frame takes about 20~50ms to decode) to keep playback smooth for a 60fps video. However, comment 38 says Chrome has no such a performance issue at all. I wonder if it is because hw acceleration is enabled on Chrome for Linux.

Hi Jya,
Can you check why video decoding is slow on the reporter's computer? Thanks!

Comment 42

[Jean-Yves Avenard [:jya]]

please provide a copy of about:support.
Also, thank you for posting the result of:
1- Install the following extension. https://addons.mozilla.org/en-US/firefox/addon/about-media/
2- Play the video
3- Open a new tab in the same video and go to about:media
4- Copy/Paste the entire content here.

Thank you.


JW, are you sure HW acceleration is enabled on Chrome, AFAIK, Google has removed the code a while back (and only VAAPI was supported).

Comment 43

[JW Wang [:jwwang] [:jw_wang]]

No, I am not sure if HW acceleration is enabled on Chrome or not. But its the only way to achieve better decoding performance as far a I know.

Comment 44

[Loic]

(In reply to JW Wang [:jwwang] [:jw_wang] from comment #43)
> No, I am not sure if HW acceleration is enabled on Chrome or not. 

It's in the page "chrome://gpu" in Chrome.

Comment 45

[6lobe]

Thanks. You can already find about:support and about:media from attachments.

Chromium does not have HW accelerated video decoding.

And here is output from chrome://gpu for the record:

Graphics Feature Status
Canvas: Hardware accelerated
Flash: Hardware accelerated
Flash Stage3D: Hardware accelerated
Flash Stage3D Baseline profile: Hardware accelerated
Compositing: Hardware accelerated
Multiple Raster Threads: Disabled
Native GpuMemoryBuffers: Software only. Hardware acceleration disabled
Rasterization: Software only. Hardware acceleration disabled
Video Decode: Software only, hardware acceleration unavailable
Video Encode: Hardware accelerated
VPx Video Decode: Software only, hardware acceleration unavailable
WebGL: Hardware accelerated
WebGL2: Hardware accelerated

Comment 46

[JW Wang [:jwwang] [:jw_wang]]

I wonder if Chrome can really play the 60fps video smoothly or it just drop frames in a less obvious way.

Please download the latest Nightly and go to about:config to add a Boolean "media.ruin-av-sync.enabled" with its value set to true.

The playback should be able to get going with some frames dropped.

Comment 47

[6lobe]

(In reply to JW Wang [:jwwang] [:jw_wang] from comment #46)
> I wonder if Chrome can really play the 60fps video smoothly or it just drop
> frames in a less obvious way.
> 
> Please download the latest Nightly and go to about:config to add a Boolean
> "media.ruin-av-sync.enabled" with its value set to true.
> 
> The playback should be able to get going with some frames dropped.

That fixes it. Playback isnt as smooth as Chromium but at least I can view the video.

Comment 48

[JW Wang [:jwwang] [:jw_wang]]

It is still worth investigation to find out why FF has inferior decoding performance. Is it possible for Chrome to use a different library to decode MP4?

Comment 49

[Jean-Yves Avenard [:jya]]

Chrome uses libvpx while we use by default ffvp9.

It could be that on this particular video libvpx performs better, though it would be extremely surprising.

You can test by setting the preference media.ffvpx.enabled to false and restart firefox.

Comment 50

[JW Wang [:jwwang] [:jw_wang]]

An mp4 file is tested in comment 40. Does ffvp9 also decode mp4?

Comment 51

[Jean-Yves Avenard [:jya]]

oh my bad... and on linux too...

so then there's no difference between what chromium is using (by default, it doesn't come with a h264 decoder btw) and what we use.

Chrome ships their own ffmpeg, we use the system one if present. So make sure what's installed on your system is a recent version.

Comment 52

[JW Wang [:jwwang] [:jw_wang]]

Per comment 41, there is no actionable item for me.

Comment 53

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Hi Anthony, this is marked as a P1. 53 will enter RC mode on coming Monday. Since we don't have as many reports of this problem nor a fix in sight, I will wontfix this for 53. Please let me know if you disagree.

Comment 54

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to Ritu Kothari (:ritu) from comment #53)
> Hi Anthony, this is marked as a P1. 53 will enter RC mode on coming Monday.
> Since we don't have as many reports of this problem nor a fix in sight, I
> will wontfix this for 53. Please let me know if you disagree.

My take on it is that it is a performance issue where Chrome goes a little bit faster. Firefox doesn't use accelerated layers on Linux and Chrome does. I'm going to change it to P3.

6lobe - you may want to try enabling gl layers by setting layers.acceleration.force-enabled=true but your mileage may vary because it was disabled due to bugs.

Comment 55

[Milan Sreckovic [:milan] (needinfo for best results)]

I run into this often enough, mostly on Facebook videos, and from home, where the internet connection isn't as fast as at work.  For all that can't reproduce - chances are you're testing on a fast connection?
I do have a (reasonably) fast computer, but the connection makes a difference.

Comment 56

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

(In reply to Milan Sreckovic [:milan] from comment #55)
> I run into this often enough, mostly on Facebook videos, and from home,
> where the internet connection isn't as fast as at work.  For all that can't
> reproduce - chances are you're testing on a fast connection?
> I do have a (reasonably) fast computer, but the connection makes a
> difference.

https://bugzilla.mozilla.org/show_bug.cgi?id=1316571#c30 says it is not about buffering. The issue is a slow machine and that Firefox is a little slower than Chrome due to lack of GL layers. The only other suggestion I've got for the reporter is to try a more recent version of ffmpeg in case that helps.

Comment 57

[Milan Sreckovic [:milan] (needinfo for best results)]

Roger that.  I was allowing for me or the reporter misdiagnosing the problem.

Comment 58

[Julien Cristau [:jcristau]]

This doesn't look like something we need to keep tracking, it's been around since 50 without a whole lot of progress.

Comment 59

[6lobe]

Setting "layers.acceleration.force-enabled=true" does not help at all.

Comment 60

[Martin Stránský [:stransky] (ni? me)]

Attachment: Bug 1316571 Disable frame drop (ruin A/V sync) if we fail to play video for some time r?alwu

Right now we do aggressive frame drop and hope a web player adjusts video quality if there's a significant frame drop.
But if a web player does not adjust video quality we fail play a clip completelly.

In this patch use a delay (5s) to give web player opportunity to update video params.
If there's still huge frame drop after this time (more than half of frames dropped) disable aggressive
frame drop to play something at least.

Comment 61

[Martin Stránský [:stransky] (ni? me)]

There's a proposal how to fix that. I can't say for Windows or Amazon prime or other services, I tested only YT but that doesn't adjust video resolution for me if there's such frame drop (at least for the 8K resolution from Bug 1750388).

I guess if we want to use such patch it should be tested which timeout is sufficient to ensure web player notices the playback difficulty - I used 5 sec now.

Comment 62

[Martin Stránský [:stransky] (ni? me)]

See https://bugzilla.mozilla.org/show_bug.cgi?id=1258870#c6 for details.