Closed Bug 633445 Opened 9 years ago Closed 3 years ago

OOM crash in nsCycleCollectingAutoRefCnt::decr(nsISupports*) or AbortIfOffMainThreadIfCheckFast (malware-related)

Categories

(Firefox :: General, defect, critical)

x86
Windows 7
defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: scoobidiver, Assigned: chofmann)

References

(Blocks 1 open bug)

Details

(Keywords: crash)

Crash Data

Attachments

(2 files)

From the landing of bug 633119, it starts showing up at #22 top crasher in 4.0b11.
It seems to be related to spywares that create aleatory dll names.

Signature	mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*)
UUID	cb6202ca-22c5-408e-a0c6-831ac2110210
Time 	2011-02-10 19:58:06.604931
Uptime	13
Last Crash	15799 seconds (4.4 hours) before submission
Install Age	59762 seconds (16.6 hours) since version was first installed.
Product	Firefox
Version	4.0b11
Build ID	20110203141415
Branch	2.0
OS	Windows NT
OS Version	6.1.7600
CPU	x86
CPU Info	GenuineIntel family 6 model 23 stepping 10
Crash Reason	EXCEPTION_BREAKPOINT
Crash Address	0x725c1a39
User Comments	
App Notes 	AdapterVendorID: 8086, AdapterDeviceID: 2a42, AdapterDriverVersion: 8.15.10.1749
xpcom_runtime_abort(###!!! ABORT: Main-thread-only object used off the main thread: file e:/builds/moz2_slave/rel-cen-w32-bld/build/xpcom/base/nsCycleCollector.cpp, line 1195)

Frame 	Module 	Signature [Expand] 	Source
0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:77
1 	mozcrt19.dll 	mozcrt19.dll@0x1327f 	
2 	xul.dll 	nsCycleCollectingAutoRefCnt::decr 	
3 	xul.dll 	nsGlobalChromeWindow::Release 	dom/base/nsGlobalWindow.cpp:1335
4 	2kWRH-5K.dll 	2kWRH-5K.dll@0x3518a 	
5 	2kWRH-5K.dll 	2kWRH-5K.dll@0x2578 	
6 	kernel32.dll 	WaitForMultipleObjectsEx 	
7 	2kWRH-5K.dll 	2kWRH-5K.dll@0x13f259 	
8 	2kWRH-5K.dll 	2kWRH-5K.dll@0x14336a 	
9 	2kWRH-5K.dll 	2kWRH-5K.dll@0x149b2f 	
10 	2kWRH-5K.dll 	2kWRH-5K.dll@0x15b45f 	
11 	ntdll.dll 	LdrpGetShimEngineInterface 	
12 	ntdll.dll 	_RtlUserThreadStart 	
13 	2kWRH-5K.dll 	2kWRH-5K.dll@0xf5a1f 	
14 	2kWRH-5K.dll 	2kWRH-5K.dll@0xf5a1f 	

More reports at:
https://crash-stats.mozilla.com/report/list?range_value=4&range_unit=weeks&signature=mozalloc_abort%28char%20const*%20const%29%20|%20mozcrt19.dll%400x1327f%20|%20nsCycleCollectingAutoRefCnt%3A%3Adecr%28nsISupports*%29
Since they are aleatory names, we probably cannot effectively blocklist them. I'm going to resolve this bug because I believe there's not a thing we can do about it.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
It is hard to blocklist, but 8H hours later, it is already #10 top crasher in 4.0b11 and I guess it will be soon #1 top crasher.
Can you add some check in nsGlobalChromeWindow::Release to prevent the crash?
No, the abort is *intentional* because all of the other options are much worse (silently leaking DOMWindows, or crashing in random places because of threadsafety issues).

I wonder if there's a really fast-spreading piece of malware out there, or whether there's just a few people crashing many many times...
Keywords: topcrash
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ]
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by a malware
> I guess it will be soon #1 top crasher.
Confirmed. It is #1 top crasher in 4.0b11 over the last 3 days.
It's impressive that either our userbase has a lot of malware installed, or that this particular piece of malware has suddenly managed to get a huge install base.

Do we have a sample of this malware? Can we go to the various antivirus vendors and get them to block this? I really cannot imagine any in-product technical solutions here.

If we have good instructions for *removing* the malware, we can email the people who have submitted crash reports with this signature and let them know how to fix it.
This shouldn't be marked "won't fix", since its affecting so many users and needs some response.  It at least is "user doc needed" and maybe some shifting around of components, to fall into evangelism/vendor outreach.

I've pinged a few contact at kasperski and symantec but we probably need a more systematic outreach plan for this one.  

I'm suspecting the instructions for removal will be too complex for a support article, but if it turns out that way we could just reference AV products that have detcted and blocked.

Also ran some reports to tell us about frequency of the .dll names reappearing.
Out of 100 reports in the sample we see only 7 reoccurring names, and none more than 3 times.

http://people.mozilla.org/crash_stacks/reports/stack-summary-mozalloc_abort.char.const..const....mozcrt19.dll.0x1327f...nsCycleCollectingAutoRefCnt::decr.nsISupports...txt
Status: RESOLVED → REOPENED
Keywords: user-doc-needed
Resolution: WONTFIX → ---
Can we tell if this exclusively is affecting 4.0bX, or are there signatures for 3.6.13 that we are seeing this under.  I didn't see any severe spiking signatures on 3.6.13 that look related in a quick check.
There is a very unusual and confusing press article at http://www.khabrein.info/news/Firefox_3_6_zero_day_exploit__Mozilla_Firefox_4_beta_11_features_Do_Not_Track_1297611686/ that mentions "On February 12th and 13th, Firefox Web browsers crashed extensively across the world",  other than that I haven't see other reports from news outlets about any outbreak targeting firefox.

input doesn't show a corresponding increase of users talking about an increase in crash comments over the last few days.  http://input.mozilla.com/en-US/beta/search?q=crash&product=firefox&version=4.0b11&date_start=&date_end=
The underlying problem exists in 3.6.x, we just didn't make it a fatal abort. This means that we'll just see crashes at random places with memory corruption or in the cycle collector.

I suspect that the repeats are probably just the same person crashing several times, and we can't assume any sort of pattern.

And since we can't fix this in the product, I don't know where you want to put the bug. Bumping to support for now!
Component: DOM → General
Product: Core → support.mozilla.com
QA Contact: general → general
Version: Trunk → unspecified
Component: General → Knowledge Base Articles
QA Contact: general → kb-articles
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by a malware → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by malware with randomly-named DLL
From a SUMO point of view, it is fixed.
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by malware with randomly-named DLL → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | MOZCRT19.dll@0x1327f ] caused by malware with randomly-named DLL
What do you mean?
> What do you mean?
No specific bug has been written for the support site (SUMO). Indeed this bug has been moved to support KB articles category. It is documented so it is fixed.
The bug here seems to be that no specific article has been written. A generic "malware may cause Firefox to crash" article is not all that useful to this specific problem.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
If any user exposed to this crash reads this report note that we are looking for samples of the suspicious .dll for the AV vendors to examine.

The .dll names are random but will look similar to these names with random combinations of numbers and letters.

oSKGqAW1J8lW.dll , oSKGqAW1J8lW.dll , W3ZWb6t7.dll , oSKGqAW1J8lW.dll , 7270fc49.dll

the .dll names might also take a form that looks like

d35d4cd0-9818-d552-4431-bca62af336b3.dll
6f0016e6-1b99-db72-8fe1-2111ef85be63.dll
bsmedberg:

What exactly do you think we should include in said article?

Do we have a workaround/solution?  Bear in mind most users can't figure out how to get a crash report, much less search for a file on their computer that has a random string.

I can put a note at the top of the Firefox crashes article saying we're being targeted by malware writers but I'm not sure that's going to solve anything.
I don't *know* yet, that's chofmann's point. We need to work with the AV vendors to identify what this thing is called, and then we can have specific instructions to give to users. Assigning to chofmann for now.
Assignee: nobody → chofmann
When there is a specific workaround/solution please start a new thread in the SUMO articles forum called "[Proposed] Name of problem" and explain what the experience is from a user's perspective and how they can fix it. Also any details about how common the problem is would be helpful.

https://support.mozilla.com/en-US/forums/knowledge-base-articles
tomcat, can you dig up a few e-mail addresses in some of the crash reports and request samples of the suspect .dll's?
There may be more than one type of DLL as well, since the amount of memory mapped seems to vary.  Looking at the ten most recent reports (giving a link to the report, the relevant Module line from the "Raw Dump" section, and the size of the memory mapping computed from its start and end given in the Module line):

bp-049731c5-493a-45ce-9a5e-819562110215
Module|abeffaec-4a9c-a4a5-51b3-a57df156601f.dll||||0x67280000|0x67542fff|0
size=0x002c3000

bp-6bc0fde4-ac7d-43d8-b29f-166c42110215
Module|-_oZb-PY-.dll||||0x20200000|0x20408fff|0
size=0x00209000

bp-40d58ace-30ec-4fa4-b0f4-3b32a2110215
Module|723868bd-51a6-f727-4912-63c209b3c0e4.dll||||0x7b660000|0x7b922fff|0
size=0x002c3000

bp-ef083880-1d5b-4e95-acac-e4d152110215
Module|94c79b00.dll||||0x4f4b0000|0x4f72dfff|0
size=0x0027e000

bp-890b081d-ad5e-4894-812c-08cc22110215
Module|90beb705-ff1f-e287-6fde-0dade1352c70.dll||||0x60fe0000|0x612aafff|0
size=0x002cb000

bp-742806e7-fe78-48bb-a8b3-4e8512110215
Module|ddbd4c6c-945d-d325-3828-987240f688fa.dll||||0x677b0000|0x67a72fff|0
size=0x002c3000

bp-9b37c018-e510-437e-b57f-618e62110215
Module|34b21fe4.dll||||0x1d040000|0x1d2b4fff|0
size=0x00275000

bp-77388882-f585-4b07-80a9-b8ef22110215
Module|caf7fad2.dll||||0x6d390000|0x6d617fff|0
size=0x00288000

bp-fcece75f-f6a4-4d86-a9d8-806fc2110215
Module|d35d4cd0-9818-d552-4431-bca62af336b3.dll||||0x7b660000|0x7b922fff|0
size=0x002c3000

bp-c41d6adf-2320-4d5c-8210-0096f2110215
Module|-_oZb-PY-.dll||||0x20200000|0x20408fff|0
size=0x00209000
(same user as second crash?)

The naming patterns of the DLLs also look a good bit different from how they looked a few days ago.
(In reply to comment #19)
> tomcat, can you dig up a few e-mail addresses in some of the crash reports and
> request samples of the suspect .dll's?

yeah will do!

cheers!
- Tomcat
As there is no antivirus category, I moved it to the add-on blockisting category.
Assignee: chofmann → nobody
Component: Knowledge Base Articles → Blocklisting
Keywords: user-doc-needed
Product: support.mozilla.com → addons.mozilla.org
QA Contact: kb-articles → blocklisting
Please stop, the only technical option we have right now is support, not blocklisting.
Component: Blocklisting → Knowledge Base Articles
Product: addons.mozilla.org → support.mozilla.com
QA Contact: blocklisting → kb-articles
Assignee: nobody → chofmann
Keywords: user-doc-needed
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | MOZCRT19.dll@0x1327f ] caused by malware with randomly-named DLL → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | NS_DebugBreak_P ] caused by malware with randomly-named DLL
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | NS_DebugBreak_P ] caused by malware with randomly-named DLL → Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by randomly-named DLL
Here are some useful comments that invalidate the virus theory:
"I don't have a virus, I've done several scans. It just gives me a stupid add and then crashes."
"Since the new upgrade to beta 12 it crashes repeatedly, when i am on Facebook and go to the game " Crime City" it crashes, i never had this problem before the update."
"everytime on facebook, when i can on a picture to view, it crashes. i'm thinking it's because of the new way facebook is displaying the photos by a pop up instead of a new page itself. the pop doesn't come up on the older firefox versions."
"youtube.com,zmovie.net,azur and asmar movie"
They don't invalidate it: e2c6a647.dll or other DLLs are still on the stack. It only means that the virus scan doesn't recognize the infection.
> It only means that the virus scan doesn't recognize the infection.
Yes if there was only the first comment.
But how can you explain that a user has no problem in 4.0b11 and has crashes in 4.0b12?
Can you link me to those reports? I couldn't find the specific comments you mention. It's possible we have another bug with the same signature, but this one is fairly well-understood.
(In reply to comment #24)
> Here are some useful comments that invalidate the virus theory:
> "I don't have a virus, I've done several scans. It just gives me a stupid add
> and then crashes."

yeah, this in particular indicates malware where the malware is showing ads.

> "Since the new upgrade to beta 12 it crashes repeatedly, when i am on Facebook
> and go to the game " Crime City" it crashes, i never had this problem before
> the update."
> "everytime on facebook, when i can on a picture to view, it crashes. i'm
> thinking it's because of the new way facebook is displaying the photos by a pop
> up instead of a new page itself. the pop doesn't come up on the older firefox
> versions."

here the infection probably happened after the installation of firefox 4, and/or the random malware .dll's don't have the same crashing incompatbility with 3.6.x and earlier version.  Probably the later if we don't seem to see this same signature with 3.6.x

> "youtube.com,zmovie.net,azur and asmar movie"

once infected the crashes could probably happend on any site.

top crashing urls are

16749 
3887 \N
 768 about:blank
 368 http://www.youtube.com/js/pyv_watch_request_ad.html
 325 http://www.facebook.com/
 313 https://www.facebook.com/login.php?login_attempt=1
 244 javascript:window.parent.rmAdIframeSrc
 244 "javascript:"""""
 208 http://www.hasznaltauto.hu/
 168 http://mail.live.com/default.aspx?wa=wsignin1.0
 120 http://www.menetrendek.hu/cgi-bin/menetrend/html.cgi
 116 http://www.google.com/
 112 http://www.orkut.com.br/Home
 107 http://d.dmcpmtrack.com/afr.php?zoneid=4&cb=123456789
> Can you link me to those reports?
It is always a randomly named dll issue. Here are comments related to the Beta 12 update:
bp-566377ea-b982-41ed-8699-93b502110226
bp-1ae0e1bd-e8cf-4ca3-a497-b480b2110226
bp-1b6fddd0-92a2-4777-ba97-071912110226
bp-e1deef76-c06b-49c6-83a8-6247c2110227 (another user)
this might be an interesting part of the attack.

one of those top crashing urls:
  http://www.youtube-nocookie.com/js/pyv_watch_request_ad.html
is one that seems to be blocked by a lot of ad blockers

the content there runs this script:

<html><head>
<script type='text/javascript'>
var google_ad_client = 'ca-pub-8174875793926223';
if (parent.yt.getConfig('PYV_YT_WEB_PROPERTY_ID')) { google_ad_client = 'ca-pub-6219811747049371'; }

if (parent.yt.getConfig('PYV_AD_HOST')) { var google_ad_host = parent.yt.getConfig('PYV_AD_HOST'); }
if (parent.yt.getConfig('PYV_AD_HOST_TIER_ID')) { var google_ad_host_tier_id = parent.yt.getConfig('PYV_AD_HOST_TIER_ID'); }
var google_max_num_ads = '1';
var google_ad_output = 'js';
var google_ad_type = 'text';
var google_only_pyv_ads = true;
var google_page_url = parent.yt.getConfig('PYV_AD_PAGE_URL') ? parent.yt.getConfig('PYV_AD_PAGE_URL') : parent.document.location;
if (parent.yt.getConfig('PYV_AD_CHANNEL')) { var google_ad_channel = parent.yt.getConfig('PYV_AD_CHANNEL'); }
if (parent.yt.getConfig('PYV_AD_KEYWORDS')) { var google_kw_type = 'broad'; var google_kw = parent.yt.getConfig('PYV_AD_KEYWORDS'); }
if (parent.yt.getConfig('PYV_AD_LANGUAGE')) { var google_language = parent.yt.getConfig('PYV_AD_LANGUAGE'); }
if (parent.yt.getConfig('PYV_AD_VIDEO_DOC_ID')) { var google_video_doc_id = parent.yt.getConfig('PYV_AD_VIDEO_DOC_ID'); }
if (parent.yt.getConfig('PYV_CAFE_EXPERIMENT_ID')) { var google_eids = [parent.yt.getConfig('PYV_CAFE_EXPERIMENT_ID')]; }

var google_ad_request_done = parent.yt.www.ads.pyv.pyvWatchAfcCallback;
</script>
<script src='http://pagead2.googlesyndication.com/pagead/show_ads.js' type='text/javascript'></script>
</head></html>


http://pagead2.googlesyndication.com/pagead/show_ads.js  seems to be obfuscated js that maybe serves ads
the id for that google_ad_client = 'ca-pub-8174875793926223'

also shows up on a number of malware control blocking lists

under this set of search results:

http://www.google.com/search?q=ca-pub-8174875793926223&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:unofficial&client=firefox-a

check out the "malware control" and snort links


and that id shows up as possible occational cross site scripting activity on facebook pages like this one

http://www.facebook.com/posted.php?id=119138201462540&start=10&hash=e76bceffd5a4810b51e5983d259a69de

down on the line <input type="hidden" id="post_form_id" name="post_form_id" ...
Depends on: 638139
we are going to start a broader e-mail campaign in Bug 638139  to try and snag a sample of the random .dll
so the attachment in Bug 638139, shows that we might have 2 or 3 different problems.

https://bug638139.bugzilla.mozilla.org/attachment.cgi?id=516347

that attachment is a list of the crashes with the signatures in the title here and a sample of those crashes where the users have supplied e-mail address with their crash report.

the sample shows any .dll in the module list that is un-versioned so that surfaces the random malware .dlls like ao4cvOn84IU-VH.dll dc51fd51.dll  thpr52-bDC7.dll and 3cbd80fe-37bf-df3c-d7f4-df4de998e612.dll etc... 

it also consistently show an un-versioned RadioWMPCore.dll UnlockerHook.dll and a few more that might be legitimate, but unversioned .dll's like our mozjs.dll which is also unversioned, but removed from the report.  even if those .dlls are legitimate, they might be hitting the same incompatibility problem seend in the malware, and could be candidates for blocking.
there are also many reports in the attachment that have no un-versioned .dlls and we need to inspect those as a third possible source of problems.
A web search gave me this slightly odd pastebin which might be relevant:

FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{7b13ec3e-999a-4b70-b9cb-2617b8323822}\components\FFExternalAlert.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{7b13ec3e-999a-4b70-b9cb-2617b8323822}\components\RadioWMPCore.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{872b5b88-9db5-4310-bdd0-ac189557e5f5}\components\FFExternalAlert.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{872b5b88-9db5-4310-bdd0-ac189557e5f5}\components\RadioWMPCore.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{bf7380fa-e3b4-4db2-af3e-9d8783a45bfc}\components\RadioWMPCore.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\{bf7380fa-e3b4-4db2-af3e-9d8783a45bfc}\components\RadioWMPCoreGecko19.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\engine@conduit.com\components\RadioWMPCore.dll
FF - component: C:\Users\Brayden\AppData\Roaming\Mozilla\Firefox\Profiles\168480wt.default\ extensions\engine@conduit.com\components\RadioWMPCoreGecko19.dll
Google search results:

{7b13ec3e-999a-4b70-b9cb-2617b8323822}: Zynga toolbar
{872b5b88-9db5-4310-bdd0-ac189557e5f5}: DVDVideoSoftTB Toolbar
{bf7380fa-e3b4-4db2-af3e-9d8783a45bfc}: uTorrentBar Toolbar

It seems likely that these are all conduit toolbars. I don't know how we'd know whether they are related to the randomly-named DLLs, it's reasonably likely that it's just a coincidence.
fligtar/jorgev, do you have any contacts a conduit that could shed light on commment 35 and comment 36?   

I guess the two questions are:

do they have some un-versioned .dlls that use the uuid for some of there toolbars where they also use the uuid for the .dll name?

do they know of any reported incompatibility problems with conduit toolbars and firefox 4 that we can help dive into?
I'm CCing the only Conduit contact we have on Bugzilla. If there's no response I'll contact some of the people I know that work there.
one more question might be is \engine@conduit.com\components\RadioWMPCore.dll a legitmate conduit component or maybe something that they are also unware of that could be being dropped into firefox installs?

we might be thinking about blocking this dll due to association with crashes and if we did we would have to block all versions since no version info is available.
It seems likely that it is a legitimate component, but you'd really have to ask Conduit to be sure. That doesn't mean it's not causing the crashes, though.
did some more checking on RadioWMPCore.dll.  it only shows up in 0-11 crashes per day for the last month as the crashing signature

https://crash-stats.mozilla.com/query/query?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=03%2F02%2F2011+15%3A36%3A58&query_search=signature&query_type=startswith&query=RadioWMPCore.dll&build_id=&process_type=any&hang_type=any&do_query=1


but its still interesting that it shows up so frequently with with the random malware .dll's around.
In reply to comment 41
> its still interesting that it shows up so frequently with with the random
> malware .dll's around.
Correlations by add-on give:
     31% (693/2272) vs.  19% (11837/62143) engine@conduit.com
So it is not fully responsible of this crash, if it is.
cww found one user with 8ca3a77e.dll and it appears it to be dropped into the product extensions directory, not the profile.

C:\Program Files\Mozilla Firefox 4.0 Beta 10\extensions\{7b661b8d-cd42-d838-c3a2-4b9f0b186ae9}\components
some of the unversioned (malware?) .dlls in these crash reports are also showing up in crashes at [@ send] Bug 614966
Blocks: 614966
I have a few more. However, everyone that's gotten back to me has either conduit toolbars or other toolbars that may be conduit toolbars but without the conduit toolbar engine.  Are we sure the malware bit is related or if it's just that users with conduit are more likely to have malware?
This is the only crash signature where Visual Basic is loaded in almost all crashes:
     95% (2786/2919) vs.   4% (2934/69862) vbscript.dll
Here are the other bugs with VB loaded in some crashes: bug 633446 (Norton IPS), and crash signatures with IE tab plugin.
Is it pertinent to blocklist VB as it is often used by malwares?
No, that's not realistic: it's just a system library and can be used for good or ill.
Might find this of interest: http://www.dslreports.com/forum/r25584604-Popup-says-qInstall-Firefox-Updateq

Link to xpi, minimal A/V detection, & basic overview of the process.

And of course you would expect the malware to get better over time.  After all, what good is it if it crashes your browser.
the initial spoofed update dialog seems to set users up to click though the xpi installs from untrusted sites as part of a "normal" update process.

looks like we could start blocking based on these addon uuids in the main install rdf, and then it sounds like there are a few more .xpi's packaged within the .xpi's.

$ unzip *.xpi
Archive:  HeterosaurusBrowze9rbnd.xpi
  inflating: HeterosaurusBrowze9r.xpi  
  inflating: conduitengine.xpi       
  inflating: browsersearch_tb.xpi    
   creating: META-INF/
  inflating: install.rdf             
host-4-24:attacks chofmann$ ls
HeterosaurusBrowze9r.xpi	HeterosaurusBrowze9rbnd.xpi	META-INF			browsersearch_tb.xpi		conduitengine.xpi		install.rdf

% cat install.rdf
<?xml version="1.0"?>

<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:em="http://www.mozilla.org/2004/em-rdf#"
     xmlns:NC="http://home.netscape.com/NC-rdf#">
        <Description about="urn:mozilla:install-manifest">
    <!-- nsIUpdateItem type for a MiX -->
    <em:type NC:parseType="Integer">32</em:type> 
                <em:id>{23b86643-7612-41f7-a516-f86a071d225d}</em:id>
    <em:name>browsersearch</em:name>
    <em:unpack>true</em:unpack>
                <em:targetApplication>
                        <Description>
                                <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
                                <em:minVersion>3.0</em:minVersion>
                                <em:maxVersion>4.0b*</em:maxVersion>                        </Description>
                </em:targetApplication>
                <!-- Flock -->
                <em:targetApplication>
                         <Description>
                                <em:id>{a463f10c-3994-11da-9945-000d60ca027b}</em:id>
                                <em:minVersion>0.4</em:minVersion>
                                <em:maxVersion>2.5.*</em:maxVersion>
                        </Description>
                </em:targetApplication>
                                
                <em:targetApplication>
                         <Description>
                                <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id>
                                <em:minVersion>0.4</em:minVersion>
                                <em:maxVersion>*</em:maxVersion>
                        </Description>
                </em:targetApplication>
                         
        </Description>          
                                
</RDF>
There are a few other xpis that got installed in addition the the ones that Chris listed in the previous comment. I will get those off of the VM in the lab. So far no crashes running with all of the xpis installed.
unpacking some of the xpi's it appears that this version of the conduit engine comes along with .xpi in comment 49

                <em:id>engine@conduit.com</em:id>
                <em:name>Conduit Engine </em:name>
    <em:unpack>true</em:unpack>
                <em:version>3.3.2.1</em:version>
In addition the 3 items that got installed first: 
HeterosaurusBrowze9r.xpi  
conduitengine.xpi       
browsersearch_tb.xpi

I got another prompt to install 2 additional items that are shown here - one is the my match community toolbar

        Heterosaurus Browze9r
        1.4.2
        true
        Demarion@www.heterosaurusbrowze9r.org

        browsersearch Community Toolbar
        3.3.2.1
        true
        {23b86643-7612-41f7-a516-f86a071d225d}

        DataMngr
        1.0
        false
        {1FD91A9C-410C-4090-BBCC-55D3450EF433}

        MediaBar
        4.1.0.00
        true
        {28387537-e3f9-4ed7-860c-11e69af4a8a0}

        Conduit Engine
        3.3.2.1
        true
        engine@conduit.com

        mymatch_v1 Community Toolbar
        3.3.2.1
        true
        {d4816b00-bba5-4d94-84bd-483bf56dbe09}
In comment 50 and comment 53, I don't see the extension id given by a user in comment 43.
The 4.0 signature for this bug is mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast
Summary: Crash [@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ][@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] caused by randomly-named DLL → Crash [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ]
preliminary analysis of some of the .dll's captured while investigating this indicates that a variation of Adware.Ezula could be running on systems encountering this crash.

more updates as we learn more about the details of the problem, and what AV solutions might offer protection.
Duplicate of this bug: 640760
An important thing: it does not happen in nightlies, even for users that have this problem in RC.
Compilation options make Firefox crashy for these randomly-named dlls. 

Here are few examples:
"I am using this version (RC 1) on Win 7 (64bit). Until recently I was using the beta 13 release and never had this problem. On my other computer, running Win 7 (32 bit) the RC 1 of firefox seems to be running fine."
bp-acc6b19c-0f42-4cae-93b0-f6ccf2110310

"this never happened to me in 4.0b13pre minefield only happens firefox 4 rc.. see ya"
bp-77594c73-5623-425a-9cdf-b56592110314
(In reply to comment #58)
> An important thing: it does not happen in nightlies, even for users that have
> this problem in RC.
> Compilation options make Firefox crashy for these randomly-named dlls. 

The compilation options ought to be identical between nightly builds and release builds, FWIW.
Minus the official branding and the update channel of course.
> Minus the official branding and the update channel of course.
Could it be caused by interferences between the update channel (checking after each startup) and these randomly-named dlls?
(In reply to comment #58)
> An important thing: it does not happen in nightlies, even for users that have
> this problem in RC.

How have you deducted this? The examples you provided can't confirm that.
And in Mozilla Crash Reports I don't find any bug report in Nightly Builds (please tell me if it's possible to get statistics about that)
> How have you deducted this?
These crash signatures haven't shown up in nightlies crash stats (see below). Two reasons are possible:
1. the nighlty user sample is not wide enough to experience these crashes.
2. it is specific to Beta and RC versions.
These crash report comments in comment 58 let me think it was the second reason.

Here are crash reports from the last 4 weeks (no 4.0b*pre):
https://crash-stats.mozilla.com/report/list?range_value=4&range_unit=weeks&signature=mozalloc_abort%28char%20const*%20const%29%20|%20NS_DebugBreak_P%20|%20nsCycleCollectingAutoRefCnt%3A%3Adecr%28nsISupports*%29
https://crash-stats.mozilla.com/report/list?range_value=4&range_unit=weeks&signature=mozalloc_abort%28char%20const*%20const%29%20|%20NS_DebugBreak_P%20|%20AbortIfOffMainThreadIfCheckFast
If this is actually true, I would wager that it's not the bits (or even the channel) that matter but the location.  For RC and release, almost everyone installs in the standard C:\Program Files\Mozilla Firefox\ but for nightlies, I think we add something to the folder name (in beta, it's Beta X).  That's probably enough to throw off malware.
just a quick update on the analysis from AV vendors.  we have one vendor working on a tool that might help to clean up a system that's been infected with this problem.   hopefully an update on that soon.
In reply to comment 64
> That's probably enough to throw off malware.
Yes, you're probably right because there is no Firefox in the program file name for nightlies: C:\Program files\Minefield.
I uploaded the so called "malware" to VirusTotal.

http://www.virustotal.com/file-scan/report.html?id=6f85c8a3cc96b254062f4d04ce089ce0f9a9d4f65325f866ace3fe3d65f06743-1303768766

It is not detected by any vendor.
Status: REOPENED → NEW
blocking2.0: --- → ?
(In reply to comment #69)
> It is not detected by any vendor.
See comment 65.
blocking2.0: ? → ---
Crash Signature: [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ]
Duplicate of this bug: 665376
With combined signatures, it is only a virtual #89 top browser crasher in 5.0.
With combined signatures, it is now #6 top browser crasher in 5.0.
Component: Knowledge Base Articles → General
Product: support.mozilla.com → Firefox
QA Contact: kb-articles → general
Crash Signature: [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ] → [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ]
Summary: Crash [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ] → OOM crash in nsCycleCollectingAutoRefCnt::decr(nsISupports*) or AbortIfOffMainThreadIfCheckFast (malware-related)
We should consider turning on the e-mail auto-responder for these signatures. See bug 670542 comment 35.
What would it say? You have malware. Sucks to be you. Is there any more information about how to get rid of it or what to do?
It could refer to the results in Virus Total.

There they can see which vendor detects that piece of Malware strain.
With combined signatures, it's only #101 top crasher in 8.0.1. I remove the topcrash and user-doc-needed keywords.
Crash Signature: [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ] → [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | nsCycleCollectingAutoRefCnt::decr(nsISupports*) ] [@ mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast ] [@ mozalloc_abort | NS_DebugBreak_P | nsCycleCollecting…
I'm marking this bug as WORKSFORME as bug crashlog signature didn't appear from a long time (over half year) [except some obsolete <16 versions, no crashes starting since 16 version].
Status: NEW → RESOLVED
Closed: 9 years ago3 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.