Enable ffmpeg on Linux platforms for media mochitests

RESOLVED FIXED in Firefox 43

Status

()

defect
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jwwang, Assigned: jya)

Tracking

unspecified
mozilla44
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox43 fixed, firefox44 fixed)

Details

Attachments

(4 attachments, 2 obsolete attachments)

(Reporter)

Description

4 years ago
We want to exercise mp4 reader/demuxer code as much as possible.
(Reporter)

Comment 1

4 years ago
Assignee: nobody → jwwang
Status: NEW → ASSIGNED
(Reporter)

Comment 2

4 years ago
Got such error messages when playing gizmo.mp4 on Try:

01:51:00     INFO -  [aac @ 0x7f3e2c2cd800] err{or,}_recognition separate: 1; 1
01:51:00     INFO -  [aac @ 0x7f3e2c2cd800] err{or,}_recognition combined: 1; 1
01:51:00     INFO -  [aac @ 0x7f3e2c2cd800] Unsupported bit depth: 0
01:51:00     INFO -  [h264 @ 0x7f3e2c2d0000] err{or,}_recognition separate: 1; 1
01:51:00     INFO -  [h264 @ 0x7f3e2c2d0000] err{or,}_recognition combined: 1; 1
01:51:00     INFO -  [h264 @ 0x7f3e2c2d0000] Unsupported bit depth: 0

Not a single video is decoded where FrameStatistics::GetParsedFrames() and FrameStatistics::GetDecodedFrames() return zeros.

Btw, I can't repro the problem on my computer where LIBAV_VER is 54 while it is 53 on Try.


Hi Edwin,
Do you have any clue about the error messages?
Flags: needinfo?(edwin)
(In reply to JW Wang [:jwwang] from comment #2)
> Do you have any clue about the error messages?

Looks like ABI incompatibility. Jean-Yves knows a lot more about this.
Flags: needinfo?(edwin) → needinfo?(jyavenard)
(Assignee)

Comment 4

4 years ago
Comment on attachment 8598517 [details] [diff] [review]
1159171_enable_ffmpeg_on_linux-v1.patch

Review of attachment 8598517 [details] [diff] [review]:
-----------------------------------------------------------------

With this patch, making FFmpeg by default will mean that on mac with ffmpeg compilation enabled, the Apple VDA and VT (hardware accelerated) will never be use anymore (as PlatformDecoder::Create() will check for FFmpeg and if found stop trying new decoders)
PlatformDecoderModule::Create() should be modified so the apple decoders are tried first, and still have an option to force the use of ffmpeg.
(Assignee)

Comment 5

4 years ago
(In reply to JW Wang [:jwwang] from comment #2)

> Btw, I can't repro the problem on my computer where LIBAV_VER is 54 while it
> is 53 on Try.

which LIBAV_VER is that? libav or ffmpeg ?
Flags: needinfo?(jwwang)
(Reporter)

Comment 6

4 years ago
Comment on attachment 8598517 [details] [diff] [review]
1159171_enable_ffmpeg_on_linux-v1.patch

Review of attachment 8598517 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/profiles/prefs_general.js
@@ +172,5 @@
>  
> +// Enable fragmented MP4 parser for testing
> +user_pref("media.fragmented-mp4.exposed", true);
> +
> +#if defined(LINUX)

Is LINUX also defined on Mac? If not, this change should affect Linux platforms only and Mac can still prefer Apple VDA and VT.
(Assignee)

Comment 7

4 years ago
(In reply to JW Wang [:jwwang] from comment #6)
> Comment on attachment 8598517 [details] [diff] [review]
> Is LINUX also defined on Mac? If not, this change should affect Linux
> platforms only and Mac can still prefer Apple VDA and VT.

oh, you're right... my bad.
(Reporter)

Comment 8

4 years ago
(In reply to Jean-Yves Avenard [:jya] from comment #5)
> which LIBAV_VER is that? libav or ffmpeg ?

https://hg.mozilla.org/mozilla-central/file/617dbce26726/dom/media/fmp4/ffmpeg/FFmpegLibs.h#l27

It is the version number of libavformat.
(Assignee)

Updated

4 years ago
Attachment #8598517 - Attachment is obsolete: true
(Assignee)

Updated

4 years ago
Assignee: jwwang → jyavenard
(Assignee)

Comment 10

4 years ago
(In reply to JW Wang [:jwwang] from comment #8)
> (In reply to Jean-Yves Avenard [:jya] from comment #5)
> > which LIBAV_VER is that? libav or ffmpeg ?
> 
> https://hg.mozilla.org/mozilla-central/file/617dbce26726/dom/media/fmp4/
> ffmpeg/FFmpegLibs.h#l27
> 
> It is the version number of libavformat.

sure... but which libavformat is it? From FFmpeg (www.ffmpeg.org) or LibAV (www.libav.org). And which version of this is it ?

ABI compatibility is a bit hard to maintain because that while similar they all use their own. Just trying to reproduce the problem locally.
(Assignee)

Comment 11

4 years ago
libavformat doesn't appear to be install on the try machine. Only gstreamer:

from mgerva:
find /builds/mock_mozilla/ -type f | grep -i libav
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/libavahi-common.so.3.5.1
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/libavahi-glib.so.1.0.1
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/libavahi-client.so.3.2.5
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/libavahi-core.so.6.0.1
/builds/mock_mozilla/mozilla-centos6-x86_64-android/root/usr/lib64/libavahi-common.so.3.5.1
/builds/mock_mozilla/mozilla-centos6-x86_64-android/root/usr/lib64/libavahi-client.so.3.2.5

[root@try-linux64-spot-144.try.releng.use1.mozilla.com ~]# find /builds/mock_mozilla/ -type f | grep -i ffm
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/include/gtk-2.0/gtk/gtktearoffmenuitem.h
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/gstreamer-0.10/libgstffmpegcolorspace.so
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/bin/tiffmedian
/builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/share/man/man1/tiffmedian.1.gz
/builds/mock_mozilla/mozilla-centos6-x86_64-android/root/usr/bin/tiffmedian
/builds/mock_mozilla/mozilla-centos6-x86_64-android/root/usr/share/man/man1/tiffmedian.1.gz

[root@try-linux64-spot-144.try.releng.use1.mozilla.com ~]# ls /builds/mock_mozilla/mozilla-centos6-x86_64/root/usr/lib64/gstreamer-0.10
libgstadder.so         libgstaudiorate.so      libgstcoreelements.so  libgstffmpegcolorspace.so  libgstogg.so       libgsttcp.so                libgstvideorate.so     libgstvorbis.so
libgstalsa.so          libgstaudioresample.so  libgstcoreindexers.so  libgstgdp.so               libgstpango.so     libgsttheora.so             libgstvideoscale.so    libgstximagesink.so
libgstapp.so           libgstaudiotestsrc.so   libgstdecodebin2.so    libgstgio.so               libgstplaybin.so   libgsttypefindfunctions.so  libgstvideotestsrc.so  libgstxvimagesink.so
libgstaudioconvert.so  libgstcdparanoia.so     libgstdecodebin.so     libgstlibvisual.so         libgstsubparse.so  libgstvideo4linux.so        libgstvolume.so
[root@try-linux64-spot-144.try.releng.use1.mozilla.com ~]#

maybe that's why it's just not working.
Now, what is bizarre is that if libavformat isn't installed, we shouldn't even reach that part of the code.
(Assignee)

Comment 12

4 years ago
Would you have a link to the try test?
(Reporter)

Comment 13

4 years ago
(In reply to Jean-Yves Avenard [:jya] from comment #10)
> (In reply to JW Wang [:jwwang] from comment #8)
> > (In reply to Jean-Yves Avenard [:jya] from comment #5)
> > > which LIBAV_VER is that? libav or ffmpeg ?
> > 
> > https://hg.mozilla.org/mozilla-central/file/617dbce26726/dom/media/fmp4/
> > ffmpeg/FFmpegLibs.h#l27
> > 
> > It is the version number of libavformat.
> 
> sure... but which libavformat is it? From FFmpeg (www.ffmpeg.org) or LibAV
> (www.libav.org). And which version of this is it ?
I don't even know both FFmepg and libAV provide libavformat.so. I have /usr/lib/x86_64-linux-gnu/libavformat.so.54.20.3 installed from libav.org.

> ABI compatibility is a bit hard to maintain because that while similar they
> all use their own. Just trying to reproduce the problem locally.
How is ABI compatibility concerned here? Can you detail the problem?

The last link is long gone. I will re-run Try to get a new link.
Flags: needinfo?(jwwang)
(Assignee)

Comment 14

4 years ago
(In reply to JW Wang [:jwwang] from comment #13)
> (In reply to Jean-Yves Avenard [:jya] from comment #10)
> > (In reply to JW Wang [:jwwang] from comment #8)
> > > (In reply to Jean-Yves Avenard [:jya] from comment #5)
> > > > which LIBAV_VER is that? libav or ffmpeg ?
> > > 
> > > https://hg.mozilla.org/mozilla-central/file/617dbce26726/dom/media/fmp4/
> > > ffmpeg/FFmpegLibs.h#l27
> > > 
> > > It is the version number of libavformat.
> > 
> > sure... but which libavformat is it? From FFmpeg (www.ffmpeg.org) or LibAV
> > (www.libav.org). And which version of this is it ?
> I don't even know both FFmepg and libAV provide libavformat.so. I have
> /usr/lib/x86_64-linux-gnu/libavformat.so.54.20.3 installed from libav.org.

libAV is a fork of FFmpeg, they both install the same library names: libavcodec, libavformat, libavutil.

> 
> > ABI compatibility is a bit hard to maintain because that while similar they
> > all use their own. Just trying to reproduce the problem locally.
> How is ABI compatibility concerned here? Can you detail the problem?

The problem is that we based on which code we are going to run on the libavformat version we find:
libavformat.53/54/55 from that we assume that libavutil and libavcodec implement a particular ABI which unfortunately is completely wrong depending on which version is installed locally: ffmpeg or libav.
Using libavformat so version was a poor choice, libavcodec would have been better choice but that one changes at a much greater rate than libavformat :(

In particular how a frame is allocated use different set of API:
av_frame_alloc, avcodec_alloc_frame or allocating on the stack and called avcodec_get_frame_defaults on it.
Those are found either in libavcodec or libavutils.

I've done my best to make a universal code and to test all versions: FFmpeg from version 0.9 to 2.4 (every public releases) and LibAV version 0.8 to version 10.0 (since I last tried, version 11 came out).

I've just configured locally libav 0.8.15 (ubuntu 12.04 uses libav 0.8.17) and I can't reproduce the issue you first mentioned.

I ran into segfault with FFmpeg 0.8, but it crashes completely outside the media code, so I'm suspecting some memory corruption somewhere.

> 
> The last link is long gone. I will re-run Try to get a new link.

I'm running one here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=492a35e88e2b
Flags: needinfo?(jyavenard)
(Assignee)

Comment 15

4 years ago
looking good so far... the failure are due to the tests now working.

do you want me to take over this bug and update all the webref test?
(Assignee)

Comment 16

4 years ago
Ok.. I can reproduce a problem locally with an old libav.

Now, the question remain on if it's a bug in libav and if we can get around it, or just have to keep the ffmpeg code disable until try machines are upgraded with a newer version.
(Reporter)

Comment 17

4 years ago
Sure. Please take this bug.
(Assignee)

Comment 18

4 years ago
Get around invalid PTS calculations in buggy LibAV 0.8.5.
Attachment #8605712 - Flags: review?(edwin)
(Assignee)

Comment 19

4 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=998cef71bc77

Still getting failure with libav 0.8.5. 

I'm afraid it's too buggy to be usable ; locally I can get it to crash rather easily. It frames with the same pts several times in a row too causing some test to fail.

May have to give up trying to get it to work with ubuntu 12.04 / libav 0.8 and wait for a newer version
(Assignee)

Updated

4 years ago
Keywords: leave-open
Component: Audio/Video → Audio/Video: Playback
(Assignee)

Updated

4 years ago
Depends on: 1207021
(Assignee)

Comment 22

4 years ago
Will continue on in bug 1206979 ; I've now identified all issues preventing us to decode with libav 0.8.5 (as used on try machines)
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Comment 23

4 years ago
Actually, no .. will need to push JW's patch here.
Blocks: 1206979
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 24

4 years ago
Attachment #8664082 - Flags: review?(karlt)
(Assignee)

Comment 25

4 years ago
When ffmpeg is enabled, it will use the FFmpeg's VPX decoder. FFmpeg appears to always buffer 15 frames before returning one (this is the same with h264) causing the waiting event to be fired much earlier than when using libvpx
Attachment #8664083 - Flags: review?(edwin)
(Assignee)

Comment 26

4 years ago
Comment on attachment 8664082 [details] [diff] [review]
[MSE] P3. Update webref results.

got a failure for the same reason as P3 of this bug.

timeout due to ffmpeg buffering much more frames than libvpx.
Attachment #8664082 - Attachment is obsolete: true
Attachment #8664082 - Flags: review?(karlt)
(Assignee)

Comment 27

4 years ago
Disabling test mediasource-append-buffer.html on linux due to TIMEOUT.
FFmpeg doesn't output anything before being fed 15 frames ; and the webm test file first cluster only has 12 frames (they only feed the first cluster).
Attachment #8664104 - Flags: review?(karlt)
Comment on attachment 8664104 [details] [diff] [review]
[MSE] P3. Update webref results.

>Bug 1159171: [MSE] P3. Update webref results. r=karlt

Please document in the commit message what change is producing different test
results.

>-      if not debug and (os == "linux") and (version == "Ubuntu 12.04") and (processor == "x86") and (bits == 32): PASS
>-      if debug and (os == "win") and (version == "5.1.2600") and (processor == "x86") and (bits == 32): PASS
>-      if not debug and (os == "linux") and (version == "Ubuntu 12.04") and (processor == "x86_64") and (bits == 64): PASS
>-      if debug and (os == "linux") and (version == "Ubuntu 12.04") and (processor == "x86_64") and (bits == 64): PASS
>-      if debug and (os == "linux") and (version == "Ubuntu 12.04") and (processor == "x86") and (bits == 32): PASS

I assume the 'debug and (os == "win")' change is not intentional.
Attachment #8664104 - Flags: review?(karlt) → review+
(Assignee)

Comment 30

4 years ago
no idea what this test is doing, but using ffvpx as decoder makes it fail with dom/media/test/test_mediarecorder_bitrate.html | seek.webm encoded@1000000=5317 > encoded@100000=5504

bug 1161276.

mreavy, what could this be the reason?

The only difference between using ffvpx over libvpx for decoding is that ffvpx buffers 15 frames before outputting a frame while libvpx will output one immediately (reading to the end of the file will cause the same amount of frames to be output).

This test is about verifying that the encoder produced data with a lower bitrate ....

I can't reproduce the problem locally, regardless of the version of ffvpx used (including 0.8.5 which is the version used by try machines)
Depends on: 1161276
Flags: needinfo?(mreavy)
(Assignee)

Comment 31

4 years ago
looks like it's just intermittent results , and my guess is that as ffvpx is significantly faster than libvpx for VP8 (like 10* as fast) it just expose a problem in the encoder (seeing that it's fed data much quicker than it used to be)
Hi Jean-Yves -- I don't see how timing could impact this.  Here's what I know: there were no tests in the tree encoding decoded video (only gUM video) before my bug landed.  MediaRecorder only supports a limited set of formats which is being fixed in bug 1182426.  I suggest you try with the patches on that bug and see if that fixes your issues.  You can also dump the resulting blobs and examine the files.  The test should dump out info with bitrates and sizes.
Flags: needinfo?(mreavy)
(Assignee)

Updated

4 years ago
No longer blocks: 1206979
Depends on: 1206979
(Assignee)

Updated

4 years ago
Blocks: ffmpeg
(Assignee)

Updated

4 years ago
Depends on: 1207442
(Assignee)

Updated

4 years ago
Keywords: leave-open
https://hg.mozilla.org/mozilla-central/rev/f4253f28d4ca
https://hg.mozilla.org/mozilla-central/rev/6419850d7cbb
https://hg.mozilla.org/mozilla-central/rev/408b38dc976d
Status: REOPENED → RESOLVED
Last Resolved: 4 years ago4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
(Assignee)

Updated

4 years ago
Blocks: 1214943
(Assignee)

Comment 35

4 years ago
Comment on attachment 8603984 [details] [diff] [review]
Enable ffmpeg on Linux platforms for media mochitests

This is an approval for all but part 2.

Approval Request Comment
[Feature/regressing bug #]: 1214943
[User impact if declined]: Will cause mochitest to fail.
[Describe test coverage new/current, TreeHerder]: In central for a few weeks.
[Risks and why]: Low, at worse mochitest failure now that we are testing that a component is properly running (it wasn't tested before)
[String/UUID change made/needed]: None
Attachment #8603984 - Flags: approval-mozilla-aurora?
Comment on attachment 8603984 [details] [diff] [review]
Enable ffmpeg on Linux platforms for media mochitests

Approved for uplift to aurora. Tests for ffmpeg. jya will do the uplifts.
Attachment #8603984 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
You need to log in before you can comment on or make changes to this bug.