Firefox Startup crash @ je_free | xul.dll@0xbb...f | nsAppShell::`. deleting destructor''(unsigned int) with various proxies

RESOLVED WORKSFORME

Status

()

Core
Memory Allocator
--
critical
RESOLVED WORKSFORME
6 years ago
10 months ago

People

(Reporter: marcia, Unassigned)

Tracking

({crash})

10 Branch
x86
Windows NT
crash
Points:
---

Firefox Tracking Flags

(firefox10+, firefox11+)

Details

(Whiteboard: [startupcrash], crash signature)

(Reporter)

Description

6 years ago
This bug was filed from the Socorro interface and is 
report bp-1f7886ab-3a2c-45f9-9a59-0f8412120107 .
============================================================= 

Seen while looking at 10.0b3 data. This crash has been present in all of the 10.0 betas, and sits right now at #13 in early B3 data. https://crash-stats.mozilla.com/report/list?signature=je_free%20|%20xul.dll@0xbb999f indicates that are some dupes in the mix so the actual count may be lower.

Will need to look at manual correlations with this one to see what is going on.


Frame 	Module 	Signature [Expand] 	Source
0 	mozutils.dll 	je_free 	memory/jemalloc/jemalloc.c:6497
1 	xul.dll 	xul.dll@0xbb999f 	
2 	xul.dll 	nsAppShell::`vector deleting destructor' 	
3 	xul.dll 	nsBaseAppShell::Release 	js/xpconnect/src/XPCJSID.cpp:320
4 	xul.dll 	nsRefPtr<nsIRunnable>::~nsRefPtr<nsIRunnable> 	obj-firefox/dist/include/nsAutoPtr.h:907
5 	xul.dll 	nsAppStartup::~nsAppStartup 	toolkit/components/startup/nsAppStartup.h:74
6 	xul.dll 	nsAppStartup::Release 	toolkit/components/startup/nsAppStartup.cpp:183
7 	xul.dll 	nsRefPtr<nsIRunnable>::~nsRefPtr<nsIRunnable> 	obj-firefox/dist/include/nsAutoPtr.h:907
8 	xul.dll 	ScopedXPCOMStartup::~ScopedXPCOMStartup 	toolkit/xre/nsAppRunner.cpp:1115
9 	xul.dll 	XRE_main 	
10 	xul.dll 	mozilla::layers::LayerUserDataSet::LayerUserDataSet 	
11 	xul.dll 	XPC_WN_CallMethod 	js/xpconnect/src/XPCWrappedNativeJSOps.cpp:1554
12 	ntdll.dll 	ZwAllocateVirtualMemory 	
13 		@0x21

Comment 1

6 years ago
It was #64 top crasher in 10.0b2.

I see various uncommon proxy DLLs: asoehook.dll, sahook.dll, snxhk.dll, visicom_antiphishing.dll, idmmkb.dll, ClinckProxy.dll, IGO32.dll, mgAdaptersProxy.dll, TrueTransparencyHook.dll, avsda.dll.
In some reports, there's no uncommon DLL.
Depends on: 711973
Summary: Firefox Startup crash [@ je_free | xul.dll@0xbb999f ] → Firefox Startup crash [@ je_free | xul.dll@0xbb999f ] with various uncommon proxies
Component: General → jemalloc
Product: Firefox → Core
QA Contact: general → jemalloc

Comment 2

6 years ago
Sitting at #13 on b3 which is much higher than b2.
Keywords: topcrash

Updated

6 years ago
No longer depends on: 711973

Comment 3

6 years ago
I am going to track this for 10.
tracking-firefox10: --- → +

Comment 4

6 years ago
CC'ing Kyle Huey in case this is somehow related to bug 414946 (should have been Mac-only though), Josh Aas to follow along in case this does end up being proxy/networking related, and Anthony/Juan to see if they can line up some testing with the software associated with DLLs in comment 1.
Keywords: qawanted
Did some Googling to track down software (versions unknown):

asoehook.dll => Norton Internet Security
sahook.dll => McAfee Site Advisor
snxhk.dll => Avast! AntiVirus / Internet Security
visicom_antiphishing.dll => Visicom Media Anti-Phishing Domain Advisor (Powered by Panda Security)
idmmkb.dll => Internet Download Manager by Tonec Inc.
clinckproxy.dll => Ideacts Innovations Pvt. Ltd.
IGO32.dll => Origin IGO by Electronic Arts
mgAdaptersProxy.dll => SweetIM for Messenger by SweetIM Technologies Ltd.
TrueTransparencyHook.dll => TrueTransparency by Lefreut Software
avsda.dll => Avira Antivirus Premium by Avira Operations GmbH & Co. KG

What's the next steps? How can we isolate and track down specific versions? Is there a dll that is reported more than others?
(Reporter)

Comment 6

6 years ago
Per triage meeting today, we will be getting correlation reports for this via Socorro.
(Reporter)

Comment 7

6 years ago
Requested a correlation report for this signature in Bug 717041.

Updated

6 years ago
Depends on: 711973

Comment 8

6 years ago
Here are the requested correlations:
  je_free | xul.dll@0xbb999f|EXCEPTION_ACCESS_VIOLATION_READ (46 crashes)
     96% (44/46) vs.  44% (8682/19911) hnetcfg.dll
     96% (44/46) vs.  44% (8728/19911) wshtcpip.dll
     96% (44/46) vs.  46% (9140/19911) comres.dll
     96% (44/46) vs.  46% (9205/19911) ws2help.dll
     96% (44/46) vs.  53% (10478/19911) mpr.dll
     78% (36/46) vs.  59% (11691/19911) secur32.dll
     17% (8/46) vs.   0% (16/19911) prio.dll
    100% (46/46) vs.  84% (16742/19911) feclient.dll
     17% (8/46) vs.   5% (1038/19911) mgAdaptersProxy.dll
    100% (46/46) vs.  89% (17652/19911) browsercomps.dll
    100% (46/46) vs.  89% (17669/19911) softokn3.dll
    100% (46/46) vs.  89% (17672/19911) firefox.exe
    100% (46/46) vs.  89% (17678/19911) xpcom.dll
    100% (46/46) vs.  89% (17735/19911) dbghelp.dll
      7% (3/46) vs.   1% (121/19911) LVPrcInj.dll
    100% (46/46) vs.  94% (18753/19911) mswsock.dll
Crash Signature: [@ je_free | xul.dll@0xbb999f] → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)]
Summary: Firefox Startup crash [@ je_free | xul.dll@0xbb999f ] with various uncommon proxies → Firefox Startup crash [@ je_free | xul.dll@0xbb999f | nsAppShell::`* deleting destructor''(unsigned int)] with various proxies

Comment 9

6 years ago
The xul address shifted in 10.0b4.

>      17% (8/46) vs.   0% (16/19911) prio.dll
Process Priority Saver (see http://www.prnwatch.com/prio.html)
>      17% (8/46) vs.   5% (1038/19911) mgAdaptersProxy.dll
SweetIM for Messenger by SweetIM Technologies Ltd (see http://sweetim.com/?mlc=EN-US&)
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;
Summary: Firefox Startup crash [@ je_free | xul.dll@0xbb999f | nsAppShell::`* deleting destructor''(unsigned int)] with various proxies → Firefox Startup crash [@ je_free | xul.dll@0xbb.99f | nsAppShell::`. deleting destructor''(unsigned int)] with various proxies

Updated

6 years ago
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip; → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;

Updated

6 years ago
Summary: Firefox Startup crash [@ je_free | xul.dll@0xbb.99f | nsAppShell::`. deleting destructor''(unsigned int)] with various proxies → Firefox Startup crash @ je_free | xul.dll@0xbb...f | nsAppShell::`. deleting destructor''(unsigned int) with various proxies

Comment 10

6 years ago
As it's a startup crash, correlations change across days. The latest ones give:
  je_free | xul.dll@0xbbc99f | nsAppShell::`vector deleting destructor''(unsigned int)|EXCEPTION_ACCESS_VIOLATION_READ (67 crashes)
     99% (66/67) vs.  57% (7652/13378) lpk.dll (Windows Language Pack)
     49% (33/67) vs.  15% (2023/13378) snxhk.dll (Avast! AntiVirus / Internet Security)
     33% (22/67) vs.   1% (189/13378) guard32.dll (Comodo Firewall / Internet Security)
      6% (4/67) vs.   0% (11/13378) prio.dll (Process Priority Saver)

Updated

6 years ago
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip; → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;

Comment 11

6 years ago
Here are 10.0.1 correlations per module over consecutive days:
* Feb 13:
     27% (32/118) vs.  14% (3564/25105) snxhk.dll (Avast! AntiVirus)
     25% (29/118) vs.  12% (3043/25105) aswJsFlt.dll (Avast! AntiVirus)
* Feb 14:
     32% (32/101) vs.  13% (5933/44591) snxhk.dll (Avast! AntiVirus)
     26% (26/101) vs.  11% (5037/44591) aswJsFlt.dll (Avast! AntiVirus)
* Feb 15:
     19% (14/75) vs.   5% (2482/49762) datamngr.dll (SearcQu Toolbar)
      8%  (6/75) vs.   1% (568/49762) Iminent.WinCore.dll (Iminent IMBooster)
      9% (7/75) vs.   4% (1769/49762) mgAdaptersProxy.dll (SweetIM)
* Feb 16:
No data
* Feb 17:
      6% (5/82) vs.   0% (148/54820) prio.dll (Prio - Priority Saver)

It would be interesting to have correlations per LSP.
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip; → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #5)
> What's the next steps? How can we isolate and track down specific versions?
> Is there a dll that is reported more than others?

Based upon Scoobidiver's correlation report, let's try reproducing this startup crash with Avast! AntiVirus.
tracking-firefox11: --- → +
Currently pulling down an XP VM to test this -- if someone else is already set up please jump in. It will take me over an hour to pull the VM.

Comment 14

6 years ago
Anthony, any luck reproducing this?
(In reply to Sheila Mooney from comment #14)
> Anthony, any luck reproducing this?

No, not yet. :/
I've tried testing this and so far have yet to crash. My basic methodology was to install a previous version of Firefox (tried 10.0, 10.0.1, 10.0b3, 10.0b5) then install Avast! Antivirus free then browse around for an hour or so, update to Firefox 10.0.2 and continue to browse. I also peppered a few restarts in there. I even tried rapidly quitting and restarting Firefox several times. All to no success -- I can't get Firefox 10.* to crash with Avast! installed.

One thing to note is that when installing Avast! it installs an "avast! WebRep" add-on. It installs version 6.0.1367 which is apparently new as of today.

I supposes it's possible that they updated the add-on and it fixed this crash though I don't know how likely that is.

I'm no expert in crash investigation, and unfortunately Marcia is away. I'd be happy to help further given the appropriate guidance.

Sorry I couldn't be of more help.

Comment 17

6 years ago
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #16) 
> I supposes it's possible that they updated the add-on and it fixed this
> crash though I don't know how likely that is.
It's not the case. See today's correlation per module for 10.0.2:
     19% (18/95) vs.  13% (7199/56926) snxhk.dll
          1% (1/95) vs.   0% (16/56926) 5.1.864.0
          1% (1/95) vs.   0% (187/56926) 6.0.1203.0
          1% (1/95) vs.   1% (811/56926) 6.0.1289.0
         16% (15/95) vs.  10% (5502/56926) 6.0.1367.0

FWIW, I have used Avast! AV for years without the Avast! WebRep add-on and haven't crashed with those signatures.
The add-on installation in browsers is managed from the Avast! AV UI. It must be noted that there's a Check Automatically New Browser Installation setting related to this add-on.
(Reporter)

Comment 18

6 years ago
I tried reproducing by installing Avast Internet suite as well as the Easypass toolbar but so far no luck yet. I installed version 6.0.1367 of avast! WebRep. I will give this another attempt tomorrow.
I went through the crash comments again but saw no new information. This crash has dropped significantly since we've released FF10, but that could be for any number of reasons.

Is there any way to dive deeper into the data and find the first date this crash was reported? It's either a regression in Firefox or something we tickled in third party software considering 100% of the crashes are on FF10.
(Reporter)

Comment 20

6 years ago
I am investigating some malware that I saw in a few reports that had more uptime. Will report back.
(In reply to Marcia Knous [:marcia] from comment #20)
> I am investigating some malware that I saw in a few reports that had more
> uptime. Will report back.

Do we still believe malware to be the major cause of this startup crasher?
(Reporter)

Comment 22

6 years ago
I don't have any solid evidence yet that malware is the cause. What I do know is this crash was likely around in 9.0.1, but with a slightly different signature: https://crash-stats.mozilla.com/report/list?signature=je_free%20|%20xul.dll@0x2eba9f%20|%20clearPrefEntry. I am going to look at some more manual correlations now - the ones from the today did not show much.

Comment 23

6 years ago
(In reply to Marcia Knous [:marcia] from comment #22)
> https://crash-stats.mozilla.com/report/list?signature=je_free%20|%20xul.
> dll@0x2eba9f%20|%20clearPrefEntry.
This one is bug 677082 with a different stack.
Whiteboard: startupcrash → [startupcrash]
(Reporter)

Comment 24

6 years ago
Latest manual correlations for largest volume signature still show a correlation to Avast:

je_free | xul.dll@0xbbc98f | nsAppShell::`scalar deleting destructor''(unsigned int)|EXCEPTION_ACCESS_VIOLATION_READ (86 crashes)
     99% (85/86) vs.  84% (61181/72500) feclient.dll
     19% (16/86) vs.  11% (8003/72500) aswJsFlt.dll
     98% (84/86) vs.  91% (65689/72500) browsercomps.dll
     19% (16/86) vs.  13% (9322/72500) snxhk.dll
    100% (86/86) vs.  94% (68486/72500) firefox.exe
    100% (86/86) vs.  94% (68503/72500) xpcom.dll

avast! Script Blocking filter library is the aswJsFlt.dll according to a Google search.

Comment 25

6 years ago
It only affects 10.0 ESR.
Keywords: topcrash
(In reply to Scoobidiver from comment #25)
> It only affects 10.0 ESR.

Does it affect all 10.0 ESR versions (ie. up to and including 10.0.5esr) or only 10.0-final ESR?

Comment 29

6 years ago
This bug is a regression in 10.0 so it's not surprising that it has appeared in all 10.0 ESR builds.
I don't see that it appeared on ALL 10.0esr builds, only 10.0.5esr builds. Where are you seeing otherwise?

Comment 31

6 years ago
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #30)
> I don't see that it appeared on ALL 10.0esr builds, only 10.0.5esr builds.
> Where are you seeing otherwise?
See https://crash-stats.mozilla.com/query/query?product=Firefox&version=Firefox%3A10.0.6esrpre&version=Firefox%3A10.0.5esr&version=Firefox%3A10.0.4esr&version=Firefox%3A10.0.3esr&version=Firefox%3A10.0.2esr&range_value=4&range_unit=weeks&query_search=signature&query_type=contains&query=je_free+|+xul.dll%400xbb&do_query=1
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip; → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;
I tried to reproduce this again but failed. Can we do some outreach to the Enterprise mailing list?

Comment 33

6 years ago
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #32)
> Can we do some outreach to the Enterprise mailing list?
Only the top-10 ESR crashers can be qualified for a fix search. This one is only #289 in 10.0.5 ESR and fixed in 17.0 ESR, so it would be a waste of time.
Given recent comments, removing qawanted. Please re-add if other leads come to light which QA can investigate.
Keywords: qawanted

Updated

2 years ago
Crash Signature: [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip; → [@ je_free | xul.dll@0xbb999f] [@ je_free | xul.dll@0xbb999f | nsAppShell::`vector deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbb999f | nsAppShell::`scalar deleting destructor''(unsigned int)] [@ je_free | xul.dll@0xbbc99f | nsAp&hellip;
I'm marking this bug as WORKSFORME as bug crashlog signature didn't appear from a long time (over half year) in Firefox (except some obsolete Fx <11).
Status: NEW → RESOLVED
Last Resolved: 10 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.