Closed Bug 981869 Opened 6 years ago Closed 5 years ago

crash in liboil-0.3.so.0.3.0@0x14e28

Categories

(Core :: Audio/Video, defect, critical)

28 Branch
x86
Linux
defect
Not set
critical

Tracking

()

VERIFIED FIXED
mozilla38
Tracking Status
firefox27 --- unaffected
firefox28 - wontfix
firefox29 - wontfix
firefox30 - wontfix
firefox35 --- wontfix
firefox36 --- wontfix
firefox37 --- verified
firefox38 --- verified
firefox39 --- verified
firefox-esr31 37+ verified

People

(Reporter: alice0775, Assigned: eflores)

References

Details

(Keywords: crash, regression)

Crash Data

Attachments

(2 files, 1 obsolete file)

This bug was filed from the Socorro interface and is 
report bp-e336367c-368b-4827-b3c0-3bd3d2140310.
=============================================================

Crash when I test Bug 981369 .

Steps To Reproduce
0. You mey need to start Firefox with new profile
      This crash happens when click "PLAY AUDIO" at first time.
1. Visit http://jamesfoster.appspot.com/static/ff_mp3_popping.html
2. Click "PLAY AUDIO".

Actual Results:
 Browser crashes several seconds later
Regression window(m-c)
Good:
http://hg.mozilla.org/mozilla-central/rev/42b2a2adda8f
Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0 ID:20131207030203
Bad:
http://hg.mozilla.org/mozilla-central/rev/b789f8ff7590
Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0 ID:20131207032302
Pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=42b2a2adda8f&tochange=b789f8ff7590
That regression window doesn't make sense, it's just ffxbld auto updates - philor is there anything about those blocklist updates that would impact this lib's functioning?
Flags: needinfo?(philringnalda)
Nope, the hsts update would only have an effect if it changed a host in there the page uses, and the page is just some data URIs; the blocklist update just disabled some extensions. I don't actually remember December 7th, but Saturday morning is an extremely unlikely time for releng to have changed anything on the build slaves. Makes no sense to me.
Flags: needinfo?(philringnalda)
yes, It is slightly difficult to reproduce the crash.
At least, 
I can reproduce the crash on the following m-i tinderbox build.
http://hg.mozilla.org/integration/mozilla-inbound/rev/ccba2b6b092d
Component: General → Video/Audio
I tried to crash at least 5 times each builds.

Regression window(m-i)
Good:
http://hg.mozilla.org/integration/mozilla-inbound/rev/921be552b22f
Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0 ID:20131206214001
Bad:
http://hg.mozilla.org/integration/mozilla-inbound/rev/33e4d9281b5d
Mozilla/5.0 (X11; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0 ID:20131206222600
Pushlog:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=921be552b22f&tochange=33e4d9281b5d
Triggered by:
33e4d9281b5d	Randell Jesup — Bug 947429: add prefs that let us disable gum video and peerconnection video r=jib
Randell, Looks like you landed this can you have a look at this?
Flags: needinfo?(rjesup)
I realize this was pinned on that checkin, but.... 

There's simply no way any of that code is invoked in that test page.  The test page is all standard dom media elements (audio), which includes MediaDecoders and playback.  The patch pointed to has no code that gets run unless you use getUserMedia or PeerConnection.

Perhaps there's a compiler/linker bug (unlikely).  Perhaps there's a timing or page layout issue (whether some pointer has a 0 or 1 in a spot.  Perhaps there's some wild write that gets triggered here - but again, the testcase doesn't ever invoke the code that changed in that push.

Note that the crash is in a thread totally outside our normal code; it's in gstreamer stuff.  Looking at the other threads, it appears the MediaDecoderStateMachine called mReader->ReadMetadata() from DecodeMetaData(), and that gstreamer code likely proxied it off to the thread that crashed.

Is this crash report from a standard nightly or inbound build?  gstreamer version?  Exact OS?   VMWare or native?  For the regression checks, mozconfig? Opt or debug?  on the regression checks, were the crashes in the same place? (please run under gdb (with gstreamer/liboil/etc symbols installed) and dump "where full" and "info threads" and attach it.)

There were a few checkins in the couple of days leading into the pushlog above that did touch content/media, including one that mentions loadedmetadata not firing (though that was an Omx change).

Chris, any thoughts?
Flags: needinfo?(rjesup) → needinfo?(cpearce)
(In reply to Randell Jesup [:jesup] from comment #7)
> Is this crash report from a standard nightly or inbound build?
  standard nightly.

> gstreamer version?
  gstreamer0.10 (0.10.36-1ubuntu1) precise.

> Exact OS?
  Ubuntu12.04.1 32bit

> VMWare or native? 
  On VMWare Player 6.0.1 , hosted on Win7Sp1 64bit.

> For the regression checks, mozconfig? Opt or debug? 
  Opt build from tinderbox m-i,
  http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/

> on the regression checks, were the crashes in the same place? 
  Crashes after 1-2 sec after click "PLAY AUDIO". No sound play.


  It crashes only in this Audio file.
  So, it is not a big deal for me.
At this point this is not a severe enough crash (and also limited to Linux) to be tracking for release blocking, but if a low risk fix is available please nominate for uplift when ready.
I don't have a clue.

I can't repro this in my VM. Edwin can you repro it on your machine?
Flags: needinfo?(cpearce)
Looking at the stack, it's probably related to the Fluendo MP3 codecs; you probably have the wrong plugin to reproduce it. Trying it now.
I've had crashes with similar stacks, let me know if you want the version of the packages I have locally.
(In reply to Paul Adenot (:padenot) from comment #12)
> I've had crashes with similar stacks, let me know if you want the version of
> the packages I have locally.

I haven't been able to reproduce this, so that would be great.
Flags: needinfo?(paul)
Attached file versions.txt
Here are the packages I have on my system. I don't have STR other than "play an mp3", I was randomly browsing and listening to music. Let me know if you need more info.
Flags: needinfo?(paul)
Duplicate of this bug: 988625
Duplicate of this bug: 988629
Assignee: nobody → edwin
Status: NEW → ASSIGNED
Comment on attachment 8402424 [details] [diff] [review]
Blacklist crashy flump3dec gstreamer plugin

Repurposed the unused plugin whitelist patch from bug 876574 to instead blacklist the flump3dec plugin.
Attachment #8402424 - Flags: review?(alessandro.d)
I have all plugins disabled, on Linux64, Firefox 29.0.1, still getting the crash every time I load the Google I/O 2014 site even with a fresh profile.

Here is my corresponding crash report:

https://crash-stats.mozilla.com/report/index/c9b0fa4b-32f2-4cf7-accb-2976a2140513
Duplicate of this bug: 1057867
Attachment #8402424 - Attachment is obsolete: true
Attachment #8402424 - Flags: review?(alessandro.d)
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Review of attachment 8562527 [details] [diff] [review]:
-----------------------------------------------------------------

green on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e2d0e6e422fb
Attachment #8562527 - Flags: review?(kinetik)
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Review of attachment 8562527 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good.

::: dom/media/gstreamer/GStreamerFormatHelper.cpp
@@ +236,5 @@
> +
> +  const gchar *factoryName =
> +    gst_plugin_feature_get_name(aFeature);
> +
> +  if (((~aTypes & FactoryTypeDecoder) && strstr(className, "Decoder")) ||

!(aTypes & FactoryTypeDecoder)

seems easier to read.

::: dom/media/gstreamer/GStreamerReader.h
@@ +24,5 @@
>  #include "MP3FrameParser.h"
>  #include "ImageContainer.h"
>  #include "nsRect.h"
>  
> +enum GstAutoplaySelectResult {

Unused?

Also (if not removed) auto_play_ vs auto_plug_ naming.

::: modules/libpref/init/all.js
@@ +306,5 @@
>  #endif
>  #endif
>  #ifdef MOZ_GSTREAMER
>  pref("media.gstreamer.enabled", true);
> +pref("media.gstreamer.disable-blacklist", false);

Would you mind inverting the name/logic for this, i.e. make it enable-blacklist with a default of true?  Not that we're super-consistent with prefs, but it'd be nice if we were.
Attachment #8562527 - Flags: review?(kinetik) → review+
https://hg.mozilla.org/mozilla-central/rev/ab6236affc72
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla38
Could you nominate this bug for uplift in aurora ? thanks
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Approval Request Comment
[Feature/regressing bug #]: GStreamer MP3 playback
[User impact if declined]: Crashing on MP3 playback on Linux
[Describe test coverage new/current, TreeHerder]: In m-c since Friday 2015-02-13 PST
[Risks and why]: MP3 playback could stop working for some Linux users
[String/UUID change made/needed]: None
Attachment #8562527 - Flags: approval-mozilla-aurora?
Duplicate of this bug: 1115792
No longer blocks: 1115792
(In reply to Edwin Flores [:eflores] [:edwin] from comment #28)
> Approval Request Comment
> [Risks and why]: MP3 playback could stop working for some Linux users

As noted in bug 1133634, some YouTube videos stop working (audio works, but there's no video) as a result of this, too. We probably need more information there before we backport this.
(In reply to Daniel Holbert [:dholbert] from comment #31)
> (In reply to Edwin Flores [:eflores] [:edwin] from comment #28)
> > Approval Request Comment
> > [Risks and why]: MP3 playback could stop working for some Linux users
> 
> As noted in bug 1133634, some YouTube videos stop working (audio works, but
> there's no video) as a result of this, too. We probably need more
> information there before we backport this.

Agreed. As this is a long standing issue, I don't know that we need to uplift this to 37 rather than letting this have the time to shake out issues and fix additional bugs before shipping in 38.
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Trying to reproduce the issue now.
Attachment #8562527 - Flags: approval-mozilla-aurora?
Flags: needinfo?(lmandel)
Flags: needinfo?(lmandel)
(In reply to Edwin Flores [:eflores] [:edwin] from comment #33)
> Trying to reproduce the issue now.

Are you still working on this? I understand that this is a more serious crash that I previously understood and that we should consider taking this fix in 37.
Flags: needinfo?(edwin)
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #36)
> (In reply to Edwin Flores [:eflores] [:edwin] from comment #33)
> > Trying to reproduce the issue now.
> 
> Are you still working on this? I understand that this is a more serious
> crash that I previously understood and that we should consider taking this
> fix in 37.

Fixed in bug 1133634.
Flags: needinfo?(edwin)
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Approval Request Comment
[Feature/regressing bug #]: GStreamer MP3 playback
[User impact if declined]: Crashing on MP3 playback on Linux
[Describe test coverage new/current, TreeHerder]: In m-c since Friday 2015-02-13 PST
[Risks and why]: MP3 playback could stop working for some Linux users
[String/UUID change made/needed]: None
Attachment #8562527 - Flags: approval-mozilla-beta?
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Taking this crash fix that has been on m-c/Aurora for 3 weeks. Beta+
Attachment #8562527 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Flags: qe-verify+
We could not reproduce the original issue. Exploratory testing for Audio and Video on Firefox 37 Beta 4 revealed no new issues. 

Removing the "qe-verify" flag as it seems there's nothing more for us to do here.
Flags: qe-verify+
Looks like we need this for ESR31 can you prepare an uplift request?
Flags: needinfo?(edwin)
Comment on attachment 8562527 [details] [diff] [review]
Blacklist flump3dec plugin, rebased

Approval request comment in comment 38
Flags: needinfo?(edwin)
Attachment #8562527 - Flags: approval-mozilla-esr31?
Attachment #8562527 - Flags: approval-mozilla-esr31? → approval-mozilla-esr31+
I reproduced the initial issue, but no reliable, only twice out of around 25 tries on a single Ubuntu 12.04 32-bit machine.
I couldn't reproduce using 31.6.0 ESR.
The crash-stats shows no new crash reports after 2015-03-06.

Alice, do you still see the crash? If not, I believe we can mark this as verified. Thank you!
Flags: needinfo?(alice0775)
verified
https://hg.mozilla.org/releases/mozilla-esr31/rev/40a8b8411a0c
Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Firefox/31.0 ID:20150325203137
https://hg.mozilla.org/releases/mozilla-release/rev/29182ac68a26
Mozilla/5.0 (X11; Linux i686; rv:37.0) Gecko/20100101 Firefox/37.0 ID:20150326190726
https://hg.mozilla.org/releases/mozilla-aurora/rev/9e57e9776571
Mozilla/5.0 (X11; Linux i686; rv:38.0) Gecko/20100101 Firefox/38.0 ID:20150330004006
https://hg.mozilla.org/mozilla-central/rev/385840329d91
Mozilla/5.0 (X11; Linux i686; rv:39.0) Gecko/20100101 Firefox/39.0 ID:20150329030238
Status: RESOLVED → VERIFIED
Flags: needinfo?(alice0775)
i'm not sure if this is going to be considered relevant enough to warrant any action, but this change:

https://hg.mozilla.org/releases/mozilla-esr31/rev/c5cb8fff6daf

causes issues with older versions of gstreamer 0.10. specifically, on RHEL 6/derivatives (CentOS, etc.) where the gstreamer version is 0.10.29. 

when firefox 31.6 is built with gstreamer enabled, the browser can crash when loading media content using gstreamer plugins:

GStreamer-CRITICAL **: gst_plugin_feature_get_name: assertion `GST_IS_PLUGIN_FEATURE (feature)' failed
Segmentation fault

apologies if the following is not explained very well, or if i have got any of this wrong:

the crash occurs in GStreamerReader.cpp:

   223   const gchar *name =
   224     gst_plugin_feature_get_name(GST_PLUGIN_FEATURE(gst_element_get_factory(aElement)));
   225 
   226   if (!strcmp(name, "uridecodebin")) {
   227     g_signal_connect(G_OBJECT(aElement), "autoplug-sort",
   228                      G_CALLBACK(GStreamerReader::ElementFilterCb), aUserData);
   229   }
   230 }

gst_plugin_feature_get_name fails, and so the strcmp fails, and the browser crashes.

i think the failure occurs when aElement is a playbin2inputselector element.

i did a bit of quick comparison with gstreamer 0.10.36, which is the version installed on debian 7, where things seem to work.

somewhere between gstreamer 0.10.29 and gstreamer 0.10.36, the input selector was removed from gst-plugins-base, which i think is why the newer 0.10 versions won't see this problem.

this change could be in version 0.10.32, based on the changelog:

2010-12-31 12:14:22 +0000 ...
...
playbin2: use input-selector from core instead of internal copy

getting GStreamerReader.cpp to check if a name has been returned seems to work around the problem, e.g.:

  if (name) {
    if (!strcmp(name, "uridecodebin")) {
      g_signal_connect(G_OBJECT(aElement), "autoplug-sort",
                       G_CALLBACK(GStreamerReader::ElementFilterCb), aUserData);
    }
  }

which stops the browser from crashing, but these error messages are still displayed in the console output:

GStreamer-CRITICAL **: gst_plugin_feature_get_name: assertion `GST_IS_PLUGIN_FEATURE (feature)' failed

the 'autoplug-sort' does also cause a warning:

GLib-GObject-WARNING **: gsignal.c:2275: signal `autoplug-sort' is invalid for instance

i haven't had a chance to look at why that may be.

anyhow, just thought this may be worth reporting.

thanks,

richard
Depends on: 1153133
I can confirm that richard rigby fix is needed to be able to use Firefox with older gstreamer versions (Ubuntu 10.04 in our case). Without this patch Firefox will just crash when playing an MP3 file (e.g. from http://hpr.dogphilosophy.net/test/) in our setup, with this patch installed the MP3 file / stream will be played correctly.
(In reply to Ignaz Forster from comment #48)
> I can confirm that richard rigby fix is needed to be able to use Firefox
> with older gstreamer versions (Ubuntu 10.04 in our case). Without this patch
> Firefox will just crash when playing an MP3 file (e.g. from
> http://hpr.dogphilosophy.net/test/) in our setup, with this patch installed
> the MP3 file / stream will be played correctly.

bug 1145230 is tracking this problem I think.
Depends on: 1145230
Duplicate of this bug: 1074399
If there's anyone here who can answer bug 1153754, comment 12, please do so. A needinfo request from 2015-07-10 has yielded no response.
This needinfo probably should have been on this bug:

bug 1153754, comment 12:
(In reply to Edwin Flores [:eflores] [:edwin] from bug 981869, comment 28)
> [Risks and why]: MP3 playback could stop working for some Linux users

Please explain what these users are supposed to do to get MP3 playback to work again.
Flags: needinfo?(edwin)
(In reply to Randell Jesup [:jesup] from comment #52)
> This needinfo probably should have been on this bug:
> 
> bug 1153754, comment 12:
> (In reply to Edwin Flores [:eflores] [:edwin] from bug 981869, comment 28)
> > [Risks and why]: MP3 playback could stop working for some Linux users
> 
> Please explain what these users are supposed to do to get MP3 playback to
> work again.

Just need a different MP3 decoder installed. There are: avdec_mp3 (in gstreamer1.0-libav), mad (in gstreamer1.0-plugins-ugly), and mpg123 (in gstreamer1.0-plugins-bad).
Flags: needinfo?(edwin)
(In reply to Edwin Flores [:eflores] [:edwin] from comment #53)
> Just need a different MP3 decoder installed. There are: avdec_mp3 (in
> gstreamer1.0-libav), mad (in gstreamer1.0-plugins-ugly), and mpg123 (in
> gstreamer1.0-plugins-bad).

Please have a look at the aforementioned bug 1153754. People on OpenSUSE say they already have Gstreamer installed and updated. You can find a list of packages at bug 1153754, comment 6. I can't tell what's missing compared to my Linux Mint setup at bug 1153754, comment 8.
You need to log in before you can comment on or make changes to this bug.