Last Comment Bug 846465 - crash in mozilla::MediaPluginHost::CreateDecoder @ libstagefright.so@0x79877 when playing mp3 files on HTC devices running JB
: crash in mozilla::MediaPluginHost::CreateDecoder @ libstagefright.so@0x79877 ...
Status: VERIFIED FIXED
: crash, regression, reproducible, topcrash
Product: Core
Classification: Components
Component: Audio/Video (show other bugs)
: 20 Branch
: ARM Android
: -- critical (vote)
: mozilla22
Assigned To: Edwin Flores [:eflores] [:edwin]
:
Mentors:
: 846892 851261 (view as bug list)
Depends on:
Blocks: 812680
  Show dependency treegraph
 
Reported: 2013-02-28 12:54 PST by Scoobidiver (away)
Modified: 2013-03-28 22:38 PDT (History)
22 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
unaffected
+
verified disabled
+
verified disabled
verified
20+
21+


Attachments
Fixy Fix (7.92 KB, patch)
2013-03-20 19:50 PDT, Edwin Flores [:eflores] [:edwin]
no flags Details | Diff | Splinter Review
Fixy Fix (7.93 KB, patch)
2013-03-20 19:52 PDT, Edwin Flores [:eflores] [:edwin]
cajbir.bugzilla: review+
mark.finkle: review+
khuey: review+
lukasblakk+bugs: approval‑mozilla‑aurora+
lukasblakk+bugs: approval‑mozilla‑beta+
Details | Diff | Splinter Review

Description Scoobidiver (away) 2013-02-28 12:54:25 PST
It's #2 top crasher in 20.0b1, 21.0a2 and 22.0a1.

Here are affected devices:
*20.0:
HTC One X 	88
HTC One S 	22
HTC EVO 	10
HTC One X+ 	9
HTC6435LVW 	6
HTC J Z321e 	3
HTC One XL 	3
HTC Butterfly 	2
Rockchip TPC97809 	1
HTC X720d 	1
HUAWEI U9510E 	1
*21.0:
HTC One X 	27
HTC EVO 	3
HUAWEI U9508 	1
HTC6435LVW 	1
*22.0:
HTC One X 	24
HTC One S 	6
HTC One X+ 	3

Reports at:
https://crash-stats.mozilla.com/report/list?product=FennecAndroid&signature=%400x0
Comment 1 Scoobidiver (away) 2013-02-28 13:40:28 PST
It first showed up in 21.0a1/20121207. It's discontinuous across builds but there are crashes at least every three builds, so the regression range might be:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=6fa6e55a93b2&tochange=739f20de3c1e
Comment 2 Aaron Train [:aaronmt] 2013-02-28 13:48:57 PST
Crashing via https://addons.mozilla.org/en-US/android/addon/flash-video-downloader-youtube ?
Comment 3 Aaron Train [:aaronmt] 2013-02-28 13:51:27 PST
Kats mentioned in channel that log-output from the reports was coming from https://mxr.mozilla.org/addons/source/6584/chrome/
Comment 4 Kartikaya Gupta (email:kats@mozilla.com) 2013-02-28 14:58:27 PST
I saw log output from that addon in one of the two crash report logcats I looked at; it may be unrelated.
Comment 5 Scoobidiver (away) 2013-02-28 22:18:23 PST
Most crash reports have no installed extensions.
Comment 6 Jim Chen [:jchen] [:darchons] 2013-03-01 08:31:16 PST
I guess crash reporter is not seeing it, but there is a reasonable stack in the raw crash dump. Here is a stack trace from the report at https://crash-stats.mozilla.com/report/index/278e3a55-ab1d-4d41-b5a2-691622130227. I looked at stacks from several reports across different devices, and they all look more or less the same.

> 0x000: pc = 0x0
> 0x000: lr = libstagefright.so + 0x79877
> 0x470: libstagefright.so + 0x7a194
> 0x484: libstagefright.so + 0x7a1ba
> 0x48c: libstagefright.so + 0xca7dc
> 0x4b8: libstagefright.so + 0x7a194
> 0x4c4: libstagefright.so + 0x72eea
> 0x4f4: libstagefright.so + 0x8ab4a (MediaExtractor::Create)
> 0x534: libomxplugin.so + 0x1dae (OmxPlugin::OmxDecoder::Init+0x56)
> 0x5ec: libomxplugin.so + 0x23de (OmxPlugin::CreateDecoder+0x1a)
> 0x5fc: libxul.so + 0x70982e (mozilla::MediaPluginHost::CreateDecoder+0x42)
> 0x624: libxul.so + 0x709be4 (mozilla::MediaPluginReader::ReadMetadata+0x28)
> 0x694: libxul.so + 0x6f5d9a (mozilla::MediaDecoderStateMachine::DecodeMetadata+0x46)
> 0x6fc: libxul.so + 0x6f81d8 (mozilla::MediaDecoderStateMachine::DecodeThreadRun+0x24)
> 0x70c: libxul.so + 0x2b7238 (nsRunnableMethodImpl::Run+0x18)
> 0x714: libxul.so + 0xab0b18 (nsThread::ProcessNextEvent+0x184)
> 0x754: libxul.so + 0xa91d48 (NS_ProcessNextEvent_P+0x14)
> 0x764: libxul.so + 0xab08f8 (nsThread::ThreadFunc+0x50)

Maybe HTC introduced some more incompatibilities in their stagefright library?
Comment 7 Scoobidiver (away) 2013-03-01 08:50:48 PST
(In reply to Jim Chen [:jchen :nchen] from comment #6)
> Maybe HTC introduced some more incompatibilities in their stagefright
> library?
Why Firefox 19 is unaffected?
Comment 8 Scoobidiver (away) 2013-03-01 08:58:19 PST
Bug 818291 is in the regression range.
Comment 9 :Ehsan Akhgari 2013-03-01 09:55:36 PST
(In reply to Scoobidiver from comment #8)
> Bug 818291 is in the regression range.

I have no idea how that patch could have caused this...
Comment 10 Aaron Train [:aaronmt] 2013-03-01 13:13:07 PST
*** Bug 846892 has been marked as a duplicate of this bug. ***
Comment 11 Todd 2013-03-01 13:37:23 PST
Bug 846892 is a duplicate and can be replicated by selecting a direct mp3 link for playback. The file shown in the screenshot is: http://traffic.libsyn.com/prestonandsteve/PnS022813.mp3 from http://prestonandsteve.libsyn.com/

Device: AT&T HTC One X+ running stock carrier Android rom 4.1.1
Comment 12 Alex Keybl [:akeybl] 2013-03-01 15:48:43 PST
Let's find a more specific regression range given the STR in comment 11.
Comment 13 Aaron Train [:aaronmt] 2013-03-07 07:53:33 PST
Mobile QA doesn't have any HTC OneX+ devices or variants. Todd, would you be willing to help us out by trying older mozilla-central-android (Nightly) builds from last December to narrow down the last known good day and the first known bad day? http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/12/
Comment 14 Aaron Train [:aaronmt] 2013-03-07 07:59:54 PST
cdouble, I think you have a One X according to (https://bugzilla.mozilla.org/show_bug.cgi?id=759945#c1)
Comment 15 Andreea Pod 2013-03-07 08:24:25 PST
I borrowed a phone outside the QA team and I found the regression window for this issue:

good build: 01.02.2012  
bad build:  02.02.2012 

possible pushlog:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d2fbc67f69f5&tochange=0352a32fde64
Comment 16 Scoobidiver (away) 2013-03-07 08:38:55 PST
Based on the STR about playing mp3 audio files, I suspect bug 455165 in that regression window.
Comment 17 Paul Adenot (:padenot) (in PTO until early September) 2013-03-07 08:46:33 PST
Nope, bug 455165 has nothing to do with mp3 (it is an ogg-only patch). Not saying that bug 455165 is not responsible, but there is things that are more likely to be the cause, like bug 805700 and bug 787228.
Comment 18 Scoobidiver (away) 2013-03-07 08:56:49 PST
(In reply to Paul Adenot (:padenot) from comment #17)
> there is things that are more likely to be the cause, like bug 805700 and bug 787228.
Theoretically, those bugs are for different Android versions. Nevertheless, they might have side effects on Jelly Bean.
Comment 19 cajbir (:cajbir) 2013-03-07 14:18:44 PST
(In reply to Aaron Train [:aaronmt] from comment #14)
> cdouble, I think you have a One X according to
> (https://bugzilla.mozilla.org/show_bug.cgi?id=759945#c1)

I don't, but Edwin and Nick Cameron does.
Comment 20 Scoobidiver (away) 2013-03-11 05:58:29 PDT
StageFright decoding is disabled by default in 20.0 for Gingerbread and Honeycomb, so backing out bug 805700 and bug 787228 from Beta wouldn't affect features and might fix those crashes.
Comment 21 Edwin Flores [:eflores] [:edwin] 2013-03-13 16:27:05 PDT
Will try to reproduce this tonight.
Comment 22 cajbir (:cajbir) 2013-03-13 16:28:38 PDT
(In reply to Scoobidiver from comment #20)
> StageFright decoding is disabled by default in 20.0 for Gingerbread and
> Honeycomb, so backing out bug 805700 and bug 787228 from Beta wouldn't
> affect features and might fix those crashes.

These bugs shouldn't affect JB devices. Is it possible the crashes have gotten more numerous due to wider adoption of the JB update on HTC devices rather than a change on our end?
Comment 23 Scoobidiver (away) 2013-03-13 16:39:09 PDT
(In reply to Chris Double (:doublec) from comment #22)
> These bugs shouldn't affect JB devices.
Theoretically. Try to find other video bugs in the regression range of comment 15.

> Is it possible the crashes have gotten more numerous due to wider adoption of the JB
> update on HTC devices rather than a change on our end?
19.0 would have been affected.
Comment 24 Aaron Train [:aaronmt] 2013-03-14 13:08:21 PDT
Confirming that I don't see this problem on a HTC Gingerbread device (Desire HD). I don't have any newer HTC devices.
Comment 25 cajbir (:cajbir) 2013-03-14 19:09:05 PDT
(In reply to Edwin Flores [:eflores] [:edwin] from comment #21)
> Will try to reproduce this tonight.

Any luck?
Comment 26 Scoobidiver (away) 2013-03-15 01:09:04 PDT
Here is the list of audio/video bugs in the regression range of comment 15: bug 455165, bug 816576, bug 787228, bug 805700, and bug 813562.
Paul, what about bug 816576?
Comment 27 Paul Adenot (:padenot) (in PTO until early September) 2013-03-15 01:32:29 PDT
Not bug 816576, because according to the stack track, at this point in the audio/video setup, haven't called yet into SoundTouch (that happens later, when the audio output is created).
Comment 28 Aaron Train [:aaronmt] 2013-03-15 07:00:21 PDT
I'd encourage Chris/Edwin or Paul to order a device to progress the state of this bug since guessing change-sets in that window doesn't seem to be yielding anything.
Comment 29 Ioana Chiorean 2013-03-15 07:09:39 PDT
(In reply to Andreea Pod from comment #15)
> I borrowed a phone outside the QA team and I found the regression window for
> this issue:
> 
> good build: 01.02.2012  
> bad build:  02.02.2012 
> 
> possible pushlog:
> http://hg.mozilla.org/mozilla-central/
> pushloghtml?fromchange=d2fbc67f69f5&tochange=0352a32fde64

This was reproduced on HTC One S (JB). 

Not reproducing on HTC Desire (2.2)- we got the following message: "Video can't be played because the file is corrupt." without crashing.
Comment 30 Scoobidiver (away) 2013-03-15 07:13:31 PDT
(In reply to Ioana Chiorean from comment #29)
> Not reproducing on HTC Desire (2.2)- we got the following message: "Video
> can't be played because the file is corrupt." without crashing.
It's bug 851261.
Comment 31 Alex Keybl [:akeybl] 2013-03-15 15:26:56 PDT
Assigning to Edwin - we're now two weeks away from release. We'd like to avoid shipping a reproducible topcrash regression, so please make this a high priority.
Comment 32 cajbir (:cajbir) 2013-03-15 15:57:04 PDT
(In reply to Aaron Train [:aaronmt] from comment #28)
> I'd encourage Chris/Edwin or Paul to order a device to progress the state of
> this bug since guessing change-sets in that window doesn't seem to be
> yielding anything.

We're waiting on Edwin responding to the needinfo. He has a device and once we can run some tests on it we should be able to resolve this pretty quickly if it is a regression.
Comment 33 Edwin Flores [:eflores] [:edwin] 2013-03-16 01:53:22 PDT
Managed to reproduce on my HTC One X; even after backing out the stagefright patches from comment 15.
Comment 34 Scoobidiver (away) 2013-03-19 10:42:05 PDT
(In reply to Edwin Flores [:eflores] [:edwin] from comment #33)
> Managed to reproduce on my HTC One X; even after backing out the stagefright
> patches from comment 15.
So the remaining suspected bugs are bug 455165, bug 816576, and bug 813562. Can you back them out one by one?
Comment 35 cajbir (:cajbir) 2013-03-19 15:07:23 PDT
(In reply to Scoobidiver from comment #34)
> So the remaining suspected bugs are bug 455165, bug 816576, and bug 813562.
> Can you back them out one by one?

The crash looks to be caused by the DataSource class on the phone having a different layout to the standard stagefright class that we are expecting. If we use Android's "CreateFromURI" to hardcode a video, the DataSource we get back works but our own DataSource crashes. This seems to indicate the issue is not a regression but rather a change in the stagefright library on the device. I doubt Jellybean ever worked on the HTC One X or it has received an update that stopped it from working.
Comment 36 Todd 2013-03-19 17:23:24 PDT
(In reply to Aaron Train [:aaronmt] from comment #13)
> Mobile QA doesn't have any HTC OneX+ devices or variants. Todd, would you be
> willing to help us out by trying older mozilla-central-android (Nightly)
> builds from last December to narrow down the last known good day and the
> first known bad day?
> http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/12/


The Nightly build from December 1st (http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/12/2012-12-01-03-08-12-mozilla-central-android/) gives an error message "Video can't be played because the file is corrupt" when attempting to play mp3 files from the website http://prestonandsteve.libsyn.com/ and podfeed.net

Attempting to play the same mp3 files in Nightly builds from December 2nd (http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/2012/12/2012-12-02-03-07-23-mozilla-central-android/) and onward causes the browser to crash.

Device: AT&T HTC One X+ running stock carrier Android 4.1.1
Comment 37 Scoobidiver (away) 2013-03-20 00:35:40 PDT
(In reply to Chris Double (:doublec) from comment #35)
> I doubt Jellybean ever worked on the HTC One X or it has received an update that 
> stopped it from working.
I has never worked (see bug 851261) but it didn't crash in 19.0.2 up to 20.0a1/20121201.
Comment 38 Edwin Flores [:eflores] [:edwin] 2013-03-20 19:50:11 PDT
Created attachment 727500 [details] [diff] [review]
Fixy Fix
Comment 39 Edwin Flores [:eflores] [:edwin] 2013-03-20 19:52:46 PDT
Created attachment 727501 [details] [diff] [review]
Fixy Fix

Fixed grammar badness in comment.
Comment 40 cajbir (:cajbir) 2013-03-20 21:14:37 PDT
Comment on attachment 727501 [details] [diff] [review]
Fixy Fix

Review of attachment 727501 [details] [diff] [review]:
-----------------------------------------------------------------

r+ for the media changes with miner string fix below. You'll need to get relevant reviews for the other files. I suggest mark.finkle for package-manifest.in and a build peer for packager.mk, toolkit.mozbuild.

::: content/media/plugins/MediaPluginHost.cpp
@@ +166,5 @@
>  
> +  nsAutoString manufacturer;
> +  rv = infoService->GetPropertyAsAString(NS_LITERAL_STRING("manufacturer"), manufacturer);
> +  if (NS_SUCCEEDED(rv)) {
> +    ALOG("Android Device is: %s", NS_LossyConvertUTF16toASCII(manufacturer).get());

That should say "Android Manufacturer is"
Comment 41 Mark Finkle (:mfinkle) (use needinfo?) 2013-03-20 22:09:42 PDT
Comment on attachment 727501 [details] [diff] [review]
Fixy Fix

package files are OK
Comment 42 Mark Finkle (:mfinkle) (use needinfo?) 2013-03-21 10:26:30 PDT
*** Bug 851261 has been marked as a duplicate of this bug. ***
Comment 43 Edwin Flores [:eflores] [:edwin] 2013-03-21 19:25:37 PDT
https://hg.mozilla.org/integration/mozilla-inbound/rev/1cb7fcc85525
Comment 44 Ed Morley [:emorley] 2013-03-22 07:02:17 PDT
https://hg.mozilla.org/mozilla-central/rev/1cb7fcc85525
Comment 45 Robert Kaiser 2013-03-22 08:03:50 PDT
As this is the #2 topcrash on 20.0b5 for Android, do we still want to get this uplifted to Beta? If not, I think we want Aurora at least.
Comment 46 Scoobidiver (away) 2013-03-22 08:17:27 PDT
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #45)
> As this is the #2 topcrash on 20.0b5 for Android, do we still want to get
> this uplifted to Beta?
As long as release drivers haven't set status-firefox20 to wontfix, it's worth requesting an uplift to Beta with the usual risk assessment.
Comment 47 Lukas Blakk [:lsblakk] use ?needinfo 2013-03-22 11:25:55 PDT
This would be great to uplift if it's low risk to take in our final beta (minimal testing/verification window) - Chris or Mark can you make the case about the risk here?
Comment 48 Edwin Flores [:eflores] [:edwin] 2013-03-25 14:11:32 PDT
Comment on attachment 727501 [details] [diff] [review]
Fixy Fix

[Approval Request Comment]
Bug caused by (feature/regressing bug #):
libstagefright
User impact if declined:
Crashing on mp3 and mp4 content on JB HTC devices
Testing completed (on m-c, etc.):
OK in Nightly
Risk to taking this patch (and alternatives if risky):
Worst case, crashy device continues to crash.
String or UUID changes made by this patch:
None
Comment 49 Lukas Blakk [:lsblakk] use ?needinfo 2013-03-25 14:14:27 PDT
Comment on attachment 727501 [details] [diff] [review]
Fixy Fix

Approving for uplift since we want to try and reduce crashes on these specific devices and the risk to anything else seems negligible so it's worth a shot. Please land to branches asap.
Comment 50 Lukas Blakk [:lsblakk] use ?needinfo 2013-03-25 14:43:29 PDT
For relnote we'll want to point to https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers#On_Android_2 (also in Google Play "what's new")
Comment 52 Catalin Suciu [:csuciu] 2013-03-26 06:33:50 PDT
On FF 20 Beta 7 & Aurora 21.0a2 (2013-03-26):
 - no longer crashes when trying to play an .mp3 file from http://prestonandsteve.libsyn.com/ but the file is not played - the player is not created

On Nightly 22.0a1 (2013-03-26) 
 - no crash and the mp3's from http://prestonandsteve.libsyn.com/ are played correctly
 
Device: HTC One S (Android 4.1.1)

Note You need to log in before you can comment on or make changes to this bug.