Closed Bug 803963 Opened 12 years ago Closed 11 years ago

crash in nsContentList::ContentAppended with Ask Toolbar bundled with Avira Antivirus

Categories

(Firefox :: Extension Compatibility, defect)

Product:
Firefox
Component:
Extension Compatibility
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

()

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox18 --- verified
firefox19 + verified
firefox20 + fixed
firefox21 --- fixed

People

(Reporter: scoobidiver, Unassigned)

Details

(Keywords: crash, reproducible, topcrash, Whiteboard: [fixed in Ask Toolbar 3.17.2])

Crash Data

Attachments

(1 file)

stack trace of an interesting assertion
15.71 KB, text/plain
Details 
It's #40 top browser crasher (#28 w/o hangs) in 16.0.1 and #90 in 17.0b1.

It's correlated to Avira and Ask Toolbar embbedded with Avira:
     99% (198/200) vs.   3% (3948/122954) toolbar@ask.com
     97% (194/200) vs.   2% (2926/122954) avsda.dll

Signature 	nsContentList::ContentAppended(nsIDocument*, nsIContent*, nsIContent*, int) More Reports Search
UUID	4c497a83-b33d-4fcb-819d-054d52121018
Date Processed	2012-10-18 01:04:07
Uptime	18
Last Crash	23 seconds before submission
Install Age	3.4 days since version was first installed.
Install Time	2012-10-14 15:10:32
Product	Firefox
Version	16.0.1
Build ID	20121010144125
Release Channel	release
OS	Windows NT
OS Version	6.1.7601 Service Pack 1
Build Architecture	x86
Build Architecture Info	GenuineIntel family 6 model 37 stepping 5
Crash Reason	EXCEPTION_ACCESS_VIOLATION_READ
Crash Address	0x1c
User Comments	I clicked on a page on Comicbookmovie.com and the site crashed several times.
App Notes 	
AdapterVendorID: 0x10de, AdapterDeviceID: 0x0a22, AdapterSubsysID: 1141174b, AdapterDriverVersion: 8.17.12.7533
D2D? D2D+ DWrite? DWrite+ D3D10 Layers? D3D10 Layers+ 
EMCheckCompatibility	True
Adapter Vendor ID	0x10de
Adapter Device ID	0x0a22
Total Virtual Memory	4294836224
Available Virtual Memory	3676008448
System Memory Use Percentage	31
Available Page File	10695479296
Available Physical Memory	4419989504

Frame 	Module 	Signature 	Source
0 	xul.dll 	nsContentList::ContentAppended 	content/base/src/nsContentList.cpp:745
1 	xul.dll 	nsNodeUtils::ContentAppended 	content/base/src/nsNodeUtils.cpp:120
2 	xul.dll 	nsINode::doInsertChildAt 	content/base/src/nsINode.cpp:1321
3 	xul.dll 	nsGenericElement::InsertChildAt 	content/base/src/nsGenericElement.cpp:2626
4 	xul.dll 	nsINode::ReplaceOrInsertBefore 	content/base/src/nsINode.cpp:1890
5 	xul.dll 	nsINode::AppendChild 	obj-firefox/dist/include/nsINode.h:504
6 	xul.dll 	nsIDOMNode_AppendChild 	obj-firefox/js/xpconnect/src/dom_quickstubs.cpp:5401
7 	mozjs.dll 	js::InvokeKernel 	js/src/jsinterp.cpp:344
8 	mozjs.dll 	js::Interpret 	js/src/jsinterp.cpp:2442
9 	mozjs.dll 	js::InvokeKernel 	js/src/jsinterp.cpp:355
10 	mozjs.dll 	js::Invoke 	js/src/jsinterp.cpp:387
11 	mozjs.dll 	JS_CallFunctionValue 	js/src/jsapi.cpp:5604
12 	xul.dll 	nsJSContext::CallEventHandler 	dom/base/nsJSEnvironment.cpp:1892
13 	xul.dll 	nsGlobalWindow::RunTimeoutHandler 	dom/base/nsGlobalWindow.cpp:9552
14 	xul.dll 	nsGlobalWindow::RunTimeout 	dom/base/nsGlobalWindow.cpp:9806
15 	xul.dll 	nsGlobalWindow::TimerCallback 	dom/base/nsGlobalWindow.cpp:10077
16 	xul.dll 	nsTimerImpl::Fire 	xpcom/threads/nsTimerImpl.cpp:473
17 	xul.dll 	nsThread::ProcessNextEvent 	xpcom/threads/nsThread.cpp:624
18 	xul.dll 	mozilla::ipc::RPCChannel::DequeueTask::`scalar deleting destructor' 	
19 	nspr4.dll 	_MD_CURRENT_THREAD 	nsprpub/pr/src/md/windows/w95thred.c:312
20 	xul.dll 	mozilla::ipc::MessagePump::Run 	ipc/glue/MessagePump.cpp:82
21 	xul.dll 	MessageLoop::RunHandler 	ipc/chromium/src/base/message_loop.cc:201
22 	xul.dll 	nsRange::ContentRemoved 	content/base/src/nsRange.cpp:571
23 	xul.dll 	nsBaseAppShell::Run 	widget/xpwidgets/nsBaseAppShell.cpp:163
24 	xul.dll 	FindSubstring<wchar_t,char> 	xpcom/string/src/nsStringObsolete.cpp:680
25 	xul.dll 	nsBlockFrame::GetLineCursor 	layout/generic/nsBlockFrame.h:369
26 	xul.dll 	nsSupportsArray::GrowArrayBy 	xpcom/ds/nsSupportsArray.cpp:133
27 	xul.dll 	nsSupportsArray::GrowArrayBy 	xpcom/ds/nsSupportsArray.cpp:132
28 	xul.dll 	nsBlockFrame::GetLineCursor 	layout/generic/nsBlockFrame.h:369

More reports at:
https://crash-stats.mozilla.com/report/list?signature=nsContentList%3A%3AContentAppended%28nsIDocument*%2C+nsIContent*%2C+nsIContent*%2C+int%29
Scoobidiver, thanks for the bug report.  I work on the Ask Toolbar, and this is news to me, especially since the toolbar itself injects no binaries into the Firefox code.  I'm going to send this up the flagpole and see what I can find.
(In reply to Alex Vincent [:WeirdAl] from comment #1)
> since the toolbar itself injects no binaries into the Firefox code.
Ask Toolbar is not only bundled with Avira so correlations suggest the problem comes from Avira or an interaction between Avira and Ask Toolbar but not Ask Toolbar alone.
It started across all versions at the beginning of October.

Correlations per version confirm it's caused by the latest version of Avira:
     99% (198/200) vs.   3% (3948/122954) toolbar@ask.com
          1% (1/200) vs.   0% (1/122954) 3.12.2.100009 
          1% (1/200) vs.   1% (1094/122954) 3.15.4.100013
         10% (20/200) vs.   0% (33/122954) 3.15.5.100015
          1% (1/200) vs.   0% (1/122954) 3.15.8.29403
         88% (175/200) vs.   0% (335/122954) 3.15.8.100015
     97% (194/200) vs.   2% (2926/122954) avsda.dll
         97% (194/200) vs.   1% (671/122954) 13.4.2.163
Oops - I forgot that for Avira we do have a plugin DLL, but that should be running inside the plugin container.  A crash there should be self-contained, and shouldn't result in this stack trace.
I have this in my debugger now.  JS stack trace:
0 anonymous() ["http://g2.gumgum.com/javascripts/ggv2.js":8]
    this = [object Object]
1 anonymous() ["http://g2.gumgum.com/javascripts/ggv2.js":8]
    this = [object Window]
2 <TOP LEVEL> ["<unknown>":0]
    <failed to get 'this' value>
I'm able to exonerate Avira code from this crash:  I generated the above crash with only the Ask toolbar (customized for Avira) in place - that is, without the Avira antivirus program installed.  Also, the Ask toolbar doesn't tell gumgum scripts to load:  that's coming directly from the webpage itself.
It's #29 top browser crasher w/o hangs in 17.0 and #59 in 18.0b1 so no longer a top crasher (arbitrary restricted to the top 20).
Keywords: topcrash
It's #12 top browser crasher in 18.0.1 and #17 in 19.0b2.

Same correlations as previously:
     99% (694/701) vs.   3% (5855/171315) toolbar@ask.com
          7% (52/701) vs.   0% (82/171315) 3.15.10.100015
          9% (61/701) vs.   0% (150/171315) 3.15.11.100015
         81% (568/701) vs.   1% (1143/171315) 3.15.13.100015
          1% (4/701) vs.   0% (17/171315) 3.15.13.33021
          1% (5/701) vs.   0% (17/171315) 3.15.5.100015
          1% (4/701) vs.   0% (16/171315) 3.15.8.100015
     96% (673/701) vs.   2% (4204/171315) avsda.dll
          0% (1/701) vs.   0% (226/171315) 10.0.0.17
          1% (5/701) vs.   0% (67/171315) 13.4.2.163
         95% (667/701) vs.   1% (2284/171315) 13.4.2.360
status-firefox18: --- → affected
status-firefox19: --- → affected
status-firefox20: --- → affected
status-firefox21: --- → affected
tracking-firefox19: --- → ?
tracking-firefox20: --- → ?
Keywords: topcrash
Version: 16 Branch → Trunk
With that many crashes, there have to be some other websites where this is happening... preferably with code that isn't obfuscated like gumgum's was.  I'll take a look at this on Tuesday, when we're back at work.
Keywords: needURLs
Keywords: needURLs
Now that we have URLs maybe qa can try to reproduce and see if we can get more to go on here.
tracking-firefox19: ? → +
tracking-firefox20: ? → +
Keywords: qawanted
QA Contact: virgil.dicu
Steps to reproduce:
1. Download Avira Antivirus from http://www.avira.com/en/avira-free-antivirus
2. Install the antivirus (make sure Avira SearchFree Toolbar and Avira SearchFree options are checked on install)
3. Start Firefox and check "Allow this installation" to the Avira SearchFree Toolbar plus Web Protection 3.15.13.
4. Click "Continue" and restart Firefox  
5. With Avira Toolbar installed open: http://wetter.msn.com/ or any other URL from comment 11.

Result:
https://crash-stats.mozilla.com/report/index/bp-4e76962c-59ff-47ac-a700-4e0372130125

Not a regression, same result with Latest Nightly, Latest Aurora, Firefox 19.0 beta 3, 18.0.1 and 4.0.1.
Keywords: qawanted
Used Windows 7 x86
Mozilla/5.0 (Windows NT 6.1; rv:18.0) Gecko/20100101 Firefox/18.0
Keywords: reproducible
Virgil, does that mean that Ask Toolbar is not involved at all? If so, we should remove it from the summary of this bug.
Looking into this: I get a ton of assertions, and then the crash:

###!!! ASSERTION: nsHTMLDocument::Close(): Trying to remove nonexistent wyciwyg channel!: 'mWyciwygChannel', file c:/ff18/mozilla/content/html/document/src/nsHTMLDocument.cpp, line 1643
###!!! ASSERTION: This is unsafe! Fix the caller! Userdata callback disabled.: 'Error', file c:/ff18/mozilla/content/base/src/nsNodeUtils.cpp, line 305
###!!! ASSERTION: This is unsafe! Fix the caller!: 'Error', file c:/ff18/mozilla/content/events/src/nsEventDispatcher.cpp, line 513
###!!! ASSERTION: Want to fire DOMNodeRemoved event, but it's not safe: '(aChild->IsNodeOfType(nsINode::eCONTENT) && static_cast<nsIContent*>(aChild)-> IsInNativeAnonymousSubtree()) || IsSafeToRunScript() || sDOMNodeRemovedSuppressCount', file c:/ff18/mozilla/content/base/src/nsContentUtils.cpp, line 3921
###!!! ASSERTION: Did we run script inappropriately?: 'aKid->GetNodeParent() == this', file c:/ff18/mozilla/content/base/src/nsINode.cpp, line 1313
###!!! ASSERTION: aRoot not an ancestor of |this|?: 'cur', file c:\ff18\fx-debug\dist\include\nsINode.h, line 1199

 	xul.dll!nsINode::GetNextSibling()  Line 1150 + 0xa bytes	C++
 	xul.dll!nsINode::GetNextNodeImpl(const nsINode * aRoot=0x15897c38, const bool aSkipChildren=false)  Line 1213 + 0x8 bytes	C++
 	xul.dll!nsINode::GetNextNode(const nsINode * aRoot=0x15897c38)  Line 1163	C++
 	xul.dll!nsContentList::ContentAppended(nsIDocument * aDocument=0x10dd9168, nsIContent * aContainer=0x15897c38, nsIContent * aFirstNewContent=0x14d15068, int aNewIndexInContainer=47)  Line 848 + 0xc bytes	C++
 	xul.dll!nsNodeUtils::ContentAppended(nsIContent * aContainer=0x15897c38, nsIContent * aFirstNewContent=0x14d15068, int aNewIndexInContainer=47)  Line 129 + 0xe6 bytes	C++


0 anonymous() ["http://db3.stj.s-msn.com/br/csl/js/1/jquery-1.3.2.min.js":19]
    this = function (E,F){return new o.fn.init(E,F)}
1 anonymous() ["http://db3.stj.s-msn.com/br/csl/js/1/jquery-1.3.2.min.js":19]
    this = function (E,F){return new o.fn.init(E,F)}
2 anonymous() ["http://db3.stj.s-msn.com/br/csl/js/1/jquery-1.3.2.min.js":19]
    this = function (E,F){return new o.fn.init(E,F)}
3 anonymous() ["http://wetter.msn.com/":101]
    this = [object Window @ 0x11a92cd0 (native @ 0x14de1348)]
Near the top of this assertion we have:

xul.dll!nsINode::RemoveChild(nsINode * aOldChild=0x0abe39e8)  Line 442 + 0x16 bytes	C++

this	0x143977e0	nsINode * const
In that frame we have this as a HEAD element, and aOldChild is a SCRIPT element with src "http://platform.twitter.com/widgets.js".

But deeper in the stack, we have:

>	xul.dll!nsINode::AppendChild(nsINode * aNewChild=0x0abe39e8, tag_nsresult * aReturn=0x003bbf28)  Line 505	C++
this	0x143977e0	nsINode * const

So there's some code telling us to remove the script element we are in the process of adding.
What is the content policy on the stack?
Yeah, someone is modifying DOM during content policy call. That is strictly prohibited
http://mxr.mozilla.org/mozilla-central/source/content/base/public/nsIContentPolicy.idl#232

Can we blocklist the addons?
(In reply to Olli Pettay [:smaug] from comment #19)
> Can we blocklist the addons?
Which add-on?
Avira and Ask toolbars (if those cause these crashes). Or if those are just .dll files, block those
libraries?
(In reply to Olli Pettay [:smaug] from comment #21)
> Avira and Ask toolbars (if those cause these crashes).
Both are widespread so we need to know which ones. Disable one of them to see if the problem still occurs.
Ask is in discussion of being blocked due to other reasons anyhow (they are violating the add-on policies), but comment #13 sounds like this might be the Avira stuff alone, and that's a DLL that's being injected. If they are violating things that are prohibited, we should still first contact them before even discussing a block.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #15)
> Virgil, does that mean that Ask Toolbar is not involved at all? If so, we
> should remove it from the summary of this bug.

Ask.com becomes the default search engine after installing the Avira Toolbar. It seems like Ask Toolbar is not involved in this.

With a clean profile I tried to install separately the Ask Toolbar from http://sp.ask.com/toolbar/install/apnasktoolbar/download.php
but it looks like this is only for Internet Explorer so I could not use this in Firefox to obtain a crash.
(In reply to Virgil Dicu [:virgil] [QA] from comment #24)
> Ask.com becomes the default search engine after installing the Avira
> Toolbar. It seems like Ask Toolbar is not involved in this.

Ah, that explains a lot (though it also points to a reason why blocking Ask is in discussion anyhow).

In any case, given this seems to be Avira and it's not an add-on but a DLL apparently, the dynamic downloadable blocklist approach wouldn't work anyhow. In any case, we should try to get a contact with them so they know and we can work with them to find a solution.
I sincerely hope it's not once again the missing UUID bump of nsIPrefBranch that causes this.
Summary: crash in nsContentList::ContentAppended with Avira and Ask Toolbar → crash in nsContentList::ContentAppended with Avira Antivirus (avsda.dll)
Reading the tealeaves, I don't think that nsIPrefBranch is related to this crash.
Hello again from Ask.  On Friday we conclusively identified the file causing the crash.  It is not the Avira DLL mentioned in this ticket.  It is instead a content policy written as a JavaScript component.  We have a potential fix which we are currently reviewing to make sure it's correct in all regards.

We understand precisely how urgent this is to fix immediately, and we'll provide more information as soon as we can.
Summary: crash in nsContentList::ContentAppended with Avira Antivirus (avsda.dll) → crash in nsContentList::ContentAppended with Ask Toolbar bundled with Avira Antivirus
FYI, our team has pushed the fix out to our Production servers.  The fix itself will take some time to propagate, but after a few days, the crash numbers should start dropping rapidly.
It's #571 browser crasher in 18.0.1 and #298 in 19.0b4 over the last three days.
Status: NEW → RESOLVED
Closed: 11 years ago
status-firefox18: affected → fixed
status-firefox19: affected → fixed
status-firefox20: affected → fixed
status-firefox21: affected → fixed
Component: DOM → Extension Compatibility
Product: Core → Firefox
Resolution: --- → FIXED
Whiteboard: [fixed in Ask Toolbar 3.17.2]
Verified the fix and no crashes found for Firefox 18.0.2, Firefox 19.0 beta 4, Latest Aurora and Latest Nightly using the steps to reproduce from comment 13 and the URLs from comment 11 for Windows 7 x86.

Mozilla/5.0 (Windows NT 6.1; rv:18.0) Gecko/20100101 Firefox/18.0
Build ID: 20130201065344
status-firefox18: fixed → verified
status-firefox19: fixed → verified
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: