1200708 - MP4 video skips frames and loses sync with audio on Linux

Status: RESOLVED WORKSFORME

(Core :: Audio/Video: Playback, defect, P5)

Description

[donrhummy]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
Build ID: 20150826023504

Steps to reproduce:

Played this video directly in Firefox Beta on Android:

https://g-design.storage.googleapis.com/production/v5/assets/g-voice-flow.mp4


Actual results:

It skipped frames so that they video was behind the audio and the video frame would freeze for a few seconds at a time.


Expected results:

It should have played without skipping frames as it does in Chrome on the same linux computer.

Comment 1

[Chris Peterson [:cpeterson]]

(In reply to donrhummy from comment #0)
> Expected results:
> 
> It should have played without skipping frames as it does in Chrome on the
> same linux computer.

When you say "on the same linux computer", do you mean you are running Chrome for Android or that you are dual-booting Android and Linux on the same mobile device?

Comment 2

[donrhummy]

(In reply to Chris Peterson [:cpeterson] from comment #1)
> (In reply to donrhummy from comment #0)
> > Expected results:
> > 
> > It should have played without skipping frames as it does in Chrome on the
> > same linux computer.
> 
> When you say "on the same linux computer", do you mean you are running
> Chrome for Android or that you are dual-booting Android and Linux on the
> same mobile device?

Neither. (Apologies, I should not have mentioned Android in this bug. I had this issue occur on BOTH an Android device and a desktop computer. I filed separate bugs for each as the Firefox Android browser has its own section in the bugs. I accidentally wrote Android here.)

For this bug, the issue was seen on Firefox on:

openSUSE 13.2
Linux 3.16.7-21 x86_64
KDE 4.14.9
Dual core AMD A4-3400 APU Radeon HD Graphics
12GB RAM (9.5 GB Free)

It ran correctly in Chrome for Linux Desktop on this computer but not in Firefox.

Comment 3

[Benjamin Chen [:bechen]]

I guess something wrong at MediaFormatReader::ShouldSkip. See Bug 1170589 comment 13.

Comment 4

[Jean-Yves Avenard [:jya]]

There's nothing wrong with MediaFormatReader::ShouldSkip 

The behaviour is different to the older MediaDecoderReader (like OMXReader) which weren't async.

somehow the comment got lost, here's what was found in MP4Reader.cpp
   // The MP4Reader doesn't do normal skip-to-next-keyframe if the demuxer
   // has exposes where the next keyframe is. We can then instead skip only
   // if the time threshold (the current playback position) is after the next
   // keyframe in the stream. This means we'll only skip frames that we have
   // no hope of ever playing.

also note that the MDSM, because MediaFormatReader is fully async doesn't call RequestVideoData with the skip to next keyframe parameter either.

Comment 5

[Jean-Yves Avenard [:jya]]

Did you enable the media.fragmented-mp4.ffmpeg.enabled pref ? (otherwise you'll be using the default gstreamer player)

Comment 6

[donrhummy]

(In reply to Jean-Yves Avenard [:jya] from comment #5)
> Did you enable the media.fragmented-mp4.ffmpeg.enabled pref ? (otherwise
> you'll be using the default gstreamer player)

I did not. But why do I need to do this to get it to play well? Chrome does not need such a flag set. What is the difference? And why is gstreamer bad, ffmpeg "good" but Mozilla isn't using that?

Comment 7

[Jean-Yves Avenard [:jya]]

You do not have to, it just makes the earlier comments about MediaFormatReader irrelevant; as you would be using gstreamer.

It's not that gstreamer is bad ; it's just that we do not have as much control on what it does and how it does it as the decoding is done by an external application.

Chrome doesn't need to set such flag because it's the default: they use ffmpeg internally. We do not use ffmpeg by default.

Comment 8

[donrhummy]

(In reply to Jean-Yves Avenard [:jya] from comment #7)
> You do not have to, it just makes the earlier comments about
> MediaFormatReader irrelevant; as you would be using gstreamer.
> 
> It's not that gstreamer is bad ; it's just that we do not have as much
> control on what it does and how it does it as the decoding is done by an
> external application.
> 
> Chrome doesn't need to set such flag because it's the default: they use
> ffmpeg internally. We do not use ffmpeg by default.

Would using ffmpeg fix this? Any reason why Mozilla chose gstreamer over ffmpeg?

Comment 9

[donrhummy]

:kentuckyfriedtakahe, :k17e why is this marked P5? Skipping frames and losing sync with audio is something that's not important and will never be implemented?

Comment 10

[Jean-Yves Avenard [:jya]]

Please try nightly, you'll find that all those problems are likely gone (those changes will be uplifted to 43 soon).

This is why it's marked P5 ; it's a gstreamer issue out of our control

Comment 11

[Jean-Yves Avenard [:jya]]

gstreamer is going in bug 1234092