Closed Bug 802355 Opened 12 years ago Closed 12 years ago

crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844"

Categories

(Core Graveyard :: Plug-ins, defect)

17 Branch
defect
Not set
blocker

Tracking

(firefox17+ fixed, firefox18+ fixed, firefox19+ verified)

RESOLVED FIXED
mozilla19
Tracking Status
firefox17 + fixed
firefox18 + fixed
firefox19 + verified

People

(Reporter: scoobidiver, Assigned: johns)

References

Details

(5 keywords, Whiteboard: [qa?])

Crash Data

Attachments

(1 file)

It started spiking from 19.0a1/20121015. The regression range for the spike is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=57304bbf9c0e&tochange=942ed5747b63
It might be a regression from bug 626245 or bug 787778.

Signature 	mozalloc_abort(char const* const) | NS_DebugBreak_P | nsRefPtr<nsHttpHandler>::nsRefPtr<nsHttpHandler>(nsHttpHandler*) | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*) More Reports Search
UUID	197b855b-ff99-45a7-bf68-e35302121016
Date Processed	2012-10-16 19:51:03
Uptime	7150
Last Crash	6.3 weeks before submission
Install Age	2.0 hours since version was first installed.
Install Time	2012-10-16 17:51:33
Product	Firefox
Version	19.0a1
Build ID	20121015030612
Release Channel	nightly
OS	Windows NT
OS Version	6.0.6001 Service Pack 1
Build Architecture	x86
Build Architecture Info	GenuineIntel family 6 model 23 stepping 10
Crash Reason	EXCEPTION_BREAKPOINT
Crash Address	0x68df1999
App Notes 	
AdapterVendorID: 0x1039, AdapterDeviceID: 0x6351, AdapterSubsysID: 08011558, AdapterDriverVersion: 7.14.10.5170
D3D10 Layers? D3D10 Layers- D3D9 Layers? D3D9 Layers- xpcom_runtime_abort(###!!! ABORT: Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844)
EMCheckCompatibility	True
Adapter Vendor ID	0x1039
Adapter Device ID	0x6351
Total Virtual Memory	2147352576
Available Virtual Memory	1679745024
System Memory Use Percentage	43
Available Page File	5151834112
Available Physical Memory	1744703488

Frame 	Module 	Signature 	Source
0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:23
1 	xul.dll 	NS_DebugBreak_P 	xpcom/base/nsDebugImpl.cpp:410
2 	xul.dll 	nsRefPtr<nsHttpHandler>::nsRefPtr<nsHttpHandler> 	obj-firefox/dist/include/nsAutoPtr.h:898
3 	xul.dll 	mozilla::plugins::PluginModuleParent::StreamCast 	dom/plugins/ipc/PluginModuleParent.cpp:844
4 	xul.dll 	mozilla::plugins::PluginModuleParent::NPP_WriteReady 	dom/plugins/ipc/PluginModuleParent.cpp:662
5 	xul.dll 	nsNPAPIPluginStreamListener::OnDataAvailable 	dom/plugins/base/nsNPAPIPluginStreamListener.cpp:562
6 	xul.dll 	nsPluginStreamListenerPeer::OnDataAvailable 	dom/plugins/base/nsPluginStreamListenerPeer.cpp:931
7 	xul.dll 	nsScriptSecurityManager::SubjectPrincipalIsSystem 	caps/src/nsScriptSecurityManager.cpp:1838
8 	xul.dll 	mozilla::net::nsHttpChannel::OnDataAvailable 	netwerk/protocol/http/nsHttpChannel.cpp:5087

More reports at:
https://crash-stats.mozilla.com/query/query?product=Firefox&version=Firefox%3A19.0a1&query_search=signature&query_type=contains&query=mozilla%3A%3Aplugins%3A%3APluginModuleParent%3A%3AStreamCast&do_query=1
It's #6 top crasher in today's build.
Keywords: topcrash
Crash Signature: _NPStream*)] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCOMPtr<nsISecurityEventSink>::nsCOMPtr<nsISecurityEventSink>(nsISecurityEventSink*) | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)] → _NPStream*)] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCOMPtr<nsISecurityEventSink>::nsCOMPtr<nsISecurityEventSink>(nsISecurityEventSink*) | mozilla::plugins::PluginModuleParent::StreamCast(_NPP* _NPStream*)] [@ mozalloc_abort | NS_Debu…
OS: Windows 7 → All
It's also #4 top crasher in 18.0a2 where it first appeared in 18.0a2/20121015. The Aurora regression range for the spike is:
http://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=0f13f4e81d94&tochange=53559d475428
So it's caused either by bug 787778 or bug 801362.
Summary: Firefox 19 crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844" → Firefox 18 crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844"
Version: 19 Branch → 18 Branch
Crash Signature: _NPStream*)] [@ mozalloc_abort | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)] → _NPStream*)] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)] [@ mozalloc_abort | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)]
A test case would be very helpful here, but I have a few ideas of what might be happening.

Why we are aborting instead of just asserting and closing the channel, however, is beyond me
Assignee: nobody → jschoenick
Status: NEW → ASSIGNED
How do you know the channel is in a useful consistent state? The abort seems perfectly reasonable to me here.
Similar to bug 795683, but with a different call trace.
One comment says:
"so something in mibew.org PHP/JS chat is crashing stock Aurora. used safe mode and it still crashed it when entering into the chat.tpl view. debugger says there's a deprecated js term."

Here are correlations per extension in 18.0a2:
  mozalloc_abort(char const* const) | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)|EXCEPTION_BREAKPOINT (168 crashes)
     32% (54/168) vs.  16% (366/2353) {d10d0bf8-f5b5-c8b4-a8b2-2b9879e08c5d} (Adblock Plus, https://addons.mozilla.org/addon/1865)
     16% (27/168) vs.   2% (47/2353) {d40f5e7b-d2cf-4856-b441-cc613eeffbe3} (BetterPrivacy, https://addons.mozilla.org/addon/6623)
     15% (25/168) vs.   2% (58/2353) {635abd67-4fe9-1b23-4f01-e679fa7484c1} (Yahoo! Toolbar, https://addons.mozilla.org/addon/2032)
     14% (24/168) vs.   4% (96/2353) {1018e4d6-728f-4b20-ad56-37578a4de76b} (Flagfox, https://addons.mozilla.org/addon/5791)
     12% (20/168) vs.   3% (67/2353) {DDC359D1-844A-42a7-9AA1-88A850A938A8} (DownThemAll!, https://addons.mozilla.org/addon/201)
     11% (18/168) vs.   2% (42/2353) donottrackplus@abine.com
     96% (161/168) vs.  87% (2053/2353) {972ce4c6-7e08-4474-a285-3208198ce6fd} (Default, https://addons.mozilla.org/addon/8150)
     10% (17/168) vs.   3% (64/2353) firefox@ghostery.com (Ghostery, https://addons.mozilla.org/addon/9609)
     13% (21/168) vs.   5% (121/2353) {D4DD63FA-01E4-46a7-B6B1-EDAB7D6AD389} (Download Statusbar, https://addons.mozilla.org/addon/26)
     10% (17/168) vs.   3% (73/2353) {a0d7ccb3-214d-498b-b4aa-0e8fda9a7bf7} (WOT, https://addons.mozilla.org/addon/3456)
     10% (16/168) vs.   3% (60/2353) {b64982b1-d112-42b5-b1e4-d3867c4533f8}
Crash Signature: _NPStream*)] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)] [@ mozalloc_abort | NS_DebugBreak_P | mozilla::plugins::PluginModuleParent::StreamCast(_NPP*, _NPStream*)] → _NPStream*)] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsRefPtr<nsNPAPIPluginInstance>::nsRefPtr<nsNPAPIPluginInstance>(nsNPAPIPluginInstance*) | mozilla::plugins::PluginModuleParent::StreamCast(_NPP* _NPStream*)] [@ mozalloc_abort(char c…
It seems it has been uplifted in 17.0 Beta 2 (currently #2 top crasher).
The Beta regression range is:
http://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=6b83222781e3&tochange=2ba4becf6e35
The only bug that belongs to the three regression ranges is bug 787778.
Summary: Firefox 18 crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844" → crash spike in mozilla::plugins::PluginModuleParent::StreamCast with abort message: "Corrupted plugin stream data.: file e:/builds/moz2_slave/m-cen-w32-ntly/build/dom/plugins/ipc/PluginModuleParent.cpp, line 844"
Version: 18 Branch → 17 Branch
Severity: critical → blocker
I'm still not able to reproduce this, but this is a speculative fix - bug 787778
lost a check for OnStartRequest failures, so we may end up trying to feed
channel data to a stream listener without a plugin.

try:
https://tbpl.mozilla.org/?tree=Try&rev=430efef625a8
Attachment #673356 - Flags: review?(joshmoz)
Will make sure we backout bug 787778 before 17.0b3
Unless John can provide a fix for the crash before Monday.
Attachment #673356 - Flags: review?(joshmoz) → review+
Comment on attachment 673356 [details] [diff] [review]
Handle failures in OnStartRequest when spawning plugins

Landed:
https://hg.mozilla.org/mozilla-central/rev/84686086802c

Will keep an eye on if this signature disappears
Attachment #673356 - Flags: checkin+
Comment on attachment 673356 [details] [diff] [review]
Handle failures in OnStartRequest when spawning plugins

There appears to be just one crash on the 21st nightly, vs numerous on the 20th

The single crash on the 21st has a different call stack and looks to be unrelated:
https://crash-stats.mozilla.com/report/index/fc835e88-deed-4584-9ced-c525e2121022

I'm fairly sure that is an unrelated stop-plugin issue (bug 767635), unrelated to 787778

[Approval Request Comment]
Regression caused by (bug #): 
bug 787778

User impact if declined: 
#6 Top crasher (non-exploitable)

Testing completed (on m-c, etc.): 
Crashes appear to have stopped after landing on m-c, but no STR for verification

Risk to taking this patch (and alternatives if risky):
Moderate-to-low, handles a otherwise unhandled failure case, but does touch re-entrance sensitive code. Alternative is backing out bug 787778.
Attachment #673356 - Flags: approval-mozilla-beta?
Attachment #673356 - Flags: approval-mozilla-aurora?
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla19
Comment on attachment 673356 [details] [diff] [review]
Handle failures in OnStartRequest when spawning plugins

Thanks for doing this so fast, we definitely have time to get this into Beta 3 and confirm that the backout option is no longer needed.  Please go ahead with uplift.
Attachment #673356 - Flags: approval-mozilla-beta?
Attachment #673356 - Flags: approval-mozilla-beta+
Attachment #673356 - Flags: approval-mozilla-aurora?
Attachment #673356 - Flags: approval-mozilla-aurora+
Blocks: 787778
Keywords: csec-dos
Keywords: needURLs
I see no crash reports for this passed October 16th. Given this is a blocker though, I'm a little concerned that we don't have steps to reproduce. Is there some focused testing QA can do around this in the next Beta to make sure we are okay? Otherwise, I guess we can mark this verified.
Whiteboard: [qa?]
(In reply to Anthony Hughes, Mozilla QA (:ashughes) from comment #15)
> I see no crash reports for this passed October 16th. Given this is a blocker
> though, I'm a little concerned that we don't have steps to reproduce. Is
> there some focused testing QA can do around this in the next Beta to make
> sure we are okay? Otherwise, I guess we can mark this verified.

Unfortunately I was unable to find a test case here - finding one would likely involve pulling up the URLs involved and finding out what plugin versions were in use -- it seems like the affected sites are using media types (e.g. audio/wav), so there's probably a specific version of a media player (vlc firefox plugin, windows media player, quicktime...) that is causing the crash.
Based on comment 15 and the fact that there are no crashes in crash reports for the last 4 weeks I'm setting this to verified.
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: