Closed Bug 528798 Opened 15 years ago Closed 6 months ago

Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted [@ (signature unavailable)] [@ @0x101c4946]

Categories

(Core :: General, defect)

Product:
Core
Component:
General
Platform:
x86
macOS
Type:
defect

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: chofmann, Unassigned)

References

Details

(Whiteboard: [crashkill][crashkill-outreach][explosive?])

Attachments

(4 files)

null sig growth for 3.5.x releases over sept oct nov
6.61 KB, text/plain
Details
null sig comments for nov 1-11 2009
513.72 KB, text/plain
Details
feed of all comments in crash reports that mention zone alarm oct-nov14
43.02 KB, text/plain
Details
all null crash trends nov/dec 2009
42.00 KB, image/png
Details
No description provided.
3.5.3 and 3.5.4 bounced along at 7-18 user comments per day for crashes reporting the null signature (\N) during september, october and november ./high-comment-sigs.sh 20091* | grep "\\N 3.5" [snip first 15 days of oct] 21 \N 3.5.3 20091017-crashdata.csv 16 \N 3.5.3 20091018-crashdata.csv 16 \N 3.5.3 20091019-crashdata.csv 15 \N 3.5.3 20091020-crashdata.csv 19 \N 3.5.3 20091021-crashdata.csv 16 \N 3.5.3 20091022-crashdata.csv 17 \N 3.5.3 20091023-crashdata.csv 18 \N 3.5.3 20091024-crashdata.csv 14 \N 3.5.3 20091025-crashdata.csv 9 \N 3.5.3 20091026-crashdata.csv 19 \N 3.5.3 20091027-crashdata.csv 9 \N 3.5.3 20091028-crashdata.csv 7 \N 3.5.4 20091028-crashdata.csv 14 \N 3.5.4 20091029-crashdata.csv 15 \N 3.5.4 20091030-crashdata.csv 7 \N 3.5.3 20091030-crashdata.csv 9 \N 3.5.4 20091031-crashdata.csv 8 \N 3.5.3 20091031-crashdata.csv 10 \N 3.5.3 20091101-crashdata.csv 9 \N 3.5.4 20091101-crashdata.csv 11 \N 3.5.4 20091102-crashdata.csv 9 \N 3.5.4 20091103-crashdata.csv 7 \N 3.5.3 20091103-crashdata.csv 11 \N 3.5.4 20091104-crashdata.csv then around the time 3.5.5 shipped we started to see a dramtic increases in users commenting when sending in these crashes 31 \N 3.5.4 20091105-crashdata.csv 116 \N 3.5.5 20091106-crashdata.csv 51 \N 3.5.3 20091106-crashdata.csv 32 \N 3.5.4 20091106-crashdata.csv 152 \N 3.5.5 20091107-crashdata.csv 51 \N 3.5.3 20091107-crashdata.csv 160 \N 3.5.5 20091108-crashdata.csv 171 \N 3.5.5 20091109-crashdata.csv 157 \N 3.5.5 20091110-crashdata.csv 168 \N 3.5.5 20091111-crashdata.csv 119 \N 3.5.5 20091112-crashdata.csv 137 \N 3.5.5 20091113-crashdata.csv 140 \N 3.5.5 20091114-crashdata.csv
maybe this just corresponds to the more general patter of users migrating or transitioning to 3.5.x the attachment needs a few more eyes looking at it. my eyes are getting red and weary. it appears that back in sept we had about 2200 \N sigs per day on 3.5.2 and 3.5.3 combined then when 3.5.4 shipped we might have jumped to around 3000 sigs per day on 3.5.2 + 3.5.3 + 3.5.4 now with 3.5.5 shipped we have about 3500 of those sigs per day. what combination's might be in play here between regressions/migrations?
here are all the comments and a possible clue at the top of the individual word parsing list beyond common words there is a high instance of comments that mention zone alarm and variations on that word. 76 ZoneAlarm 74 Zone 41 zone 23 Zonealarm 15 zonealarm 4 ZoneAlarm. 3 Zonelabs 3 ZoneAlarm's 3 (Zone 2 zonelabs 2 ZoneAlarm, 2 ""Zone 1 zoneyalarm 1 zonelaps 1 zoneAlarm 1 zone, 1 zONE 1 Zonealrm 1 Zonealarm. 1 Zonealarm, 1 Zonealarm's 1 ZoneLabs' 1 ZoneLabs 1 ZoneAlarms 1 "zone 1 "ZoneAlarm's 1 ""zone 1 ""ZoneAlarme 1 ""ZoneAlarm
http://crash-stats.mozilla.com/report/index/474e21ff-3b93-4b1a-a3aa-0d4e92091105 ZoneAlarm Extreme Security Forcefield kills Firefox 3.5.5. | Version 3.5.4 worked fine. I'm using ZoneAlarm Extreme Security version 9.0.114.000, ForeField 1.5.36.15. If I | turn off ForceField then the crashing stops. Firefox 3.5.5 \N \N http://crash-stats.mozilla.com/report/index/0e8e08dc-490f-45e9-ac68-b2c4e2091106 After update to 355 crash under zonealarm force field. Stopping forcefield then works o.k. Firefox 3.5.5 \N \N http://crash-stats.mozilla.com/report/index/1f5ba4e0-9b71-409d-a571-6ec442091106 Oorzaak Forcefield van ZoneAlarm Extreme security 9.1 Firefox 3.5.5 \N \N http://crash-stats.mozilla.com/report/index/4677dc31-3e53-4d8a-a347-4e9e02091106 This is after I loaded ZoneAlarm Extreme Security version 9.1.008.000 Firefox 3.5.5 \N \N
Summary: Spike in comments submitted for crash [@ \N ] just after 3.5.5 shipped → Spike in comments submitted for crash [@ \N ] just after 3.5.5 shipped ; compat problem with zone alarm?
Whiteboard: [crashkill][crashkill-outreach][explosive?]
kev, know anyone at zone alarm?
processor notes from that first crash report in comment 4 tell a sad tale of corruption and mystery. Processor Notes WARNING: Json file missing Add-ons; /home/processor/stackwalk/bin/stackwalk.sh returned no header lines for reportid: 70038683; No thread was identified as the cause of the crash; No signature could be created because we do not know which thread crashed; /home/processor/stackwalk/bin/stackwalk.sh returned no frame lines for reportid: 70038683; /home/processor/stackwalk/bin/stackwalk.sh failed with return code 139 when processing dump 474e21ff-3b93-4b1a-a3aa-0d4e92091105
murali will like the comment word parsing in comment 3. we have talked about trying to get that running on several feedback sources. here is a case where its helping to diagnose at least part of a top crash bug in 3.5.5.
to get the full list of words and their frequency grab the attachment in comment 3 then run grep -v http null-sig-comments-nov2009.txt | awk '$0 ~ /[A-z]/' | awk '{for ( i=1 ; i <= NF ; i++ ) print $i}' | sort | uniq -c | sort -nr | more I didn't spot any other suspects but maybe others might. grep -v http null-sig-comments-nov2009.txt | awk '$0 ~ /[A-z]/' | awk '{for ( i=1 ; i <= NF ; i++ ) print $i}' | sort | uniq -c | sort -nr | grep -i zone was used to extract the zone alarm list in comment 3. variations of the zone alarm product naming info and version also turn up as outliers in the normal day-to-day word patterns and counts we might in most crash reports. 42 ForceField 42 Extreme
adding user-doc-needed http://crash-stats.mozilla.com/report/index/8ff03ca1-ec87-4bdb-a7fb-de7aa2091105 i do not know why firefox became very slow and most of the time crashes - however, i have remove program from my computer - but still am facing same problem . i though from zoneyalarm but is not - because i have try it with IE8 - and it works in probper way - please i need to know if there is any special setting that i have to run it with zone alarm extreme version. Firefox 3.5.5 \N \N
Keywords: user-doc-needed
there are several forum posts, some maybe related to this problem of being able to start, and others might be talk about other zone alarm compat problems in current and past releases of zone alarm and firefox. http://support.mozilla.com/tiki-newsearch.php?locale=en-US&q=zone+alarm&sa= I didn't see any kb articles.
here is another slice of the data stack-sig firefox-version process-time comment the whole month of oct we got 6 zone alarm comments, then big ramp in nov. there are a few other signatures where we are getting zone alarm comments but its mostly \N. a few different flash addresses top the list of other signatures. 223 \N 7 JVM_RaiseSignal 3 arena_dalloc_small | arena_dalloc | free | XPT_DestroyArena 3 NPSWF32.dll@0x2d610 3 GlobalFindAtomA 3 @0x0 | CClassCache::CDllFnPtrMoniker::BindToObjectNoSwitch(_GUID const&, void**) 2 nsIFrame::GetAncestorWithView() 1 xul.dll@0x354140 1 strcmp | PREF_UnregisterCallback 1 sqlite3VdbeIdxRowidLen 1 nsXULDocument::ResumeWalk() 1 nsPIDOMWindow::GetFrameElementInternal() 1 nsMenuPopupFrame::HidePopup(int, nsPopupState) 1 nsMappedAttributes::IndexOfAttr(nsIAtom*, int) const 1 nsINode::GetOwnerDoc() 1 nsHttpsHandler::GetProtocolFlags(unsigned int*) 1 libclient.dylib@0x1192e9 1 libavcodec_plugin.dll@0x2a2c0b 1 js_Interpret 1 imgContainer::GetFrameAt(unsigned int, gfxIImageFrame**) 1 arena_dalloc_small | arena_dalloc | free | CloseDir 1 ad2mpegin.dll@0x180fe 1 _ftol2_pentium4 1 _ftol2 1 __ClientMonitorEnumProc 1 Vcam.ax@0x3975 1 RtlpWaitForCriticalSection 1 RtlReAllocateHeap 1 NPSWF32.dll@0x4e92a 1 NPSWF32.dll@0x3f154 1 NPSWF32.dll@0x3cfde 1 NPSWF32.dll@0x2ce8b7 1 NPSWF32.dll@0x2ad7b 1 NPSWF32.dll@0x1e7e52 1 NPSWF32.dll@0x150c54 1 NPSWF32.dll@0x13c4f3 1 MOZ_PNG_progressive_combine_row 1 JS_TraceChildren 1 Flash Player@0x92054 1 Flash Player@0x2bd3b0 1 CoreFoundation@0x11ca37 1 CFReadStreamGetStatus 1 @0x1444cab 1 @0x0 | js_GetPropertyHelper
ah, the filter needs to be tightend up a bit. the simple [Zz][Oo][Nn][Ee] search is also picking up unrelated signatures and comments like GlobalFindAtomA 3.5.5 200911100506 when i type in www.floorzone.uk.com , mozzila crashes , | mozzilla has started to crash very frequently , recently over the last 4 days - it did not used to crash GlobalFindAtomA 3.5.5 200911100516 mozzila keeps crashing tried url = www.floorzone.uk.com NPSWF32.dll@0x2d610 3.5.5 200911100555 pourquoi ne puis-je acceder à la feuille de paiement amazone .fr???? NPSWF32.dll@0x2d610 3.5.5 200911100559 je voudrai savoir pourquoi je ne puis commander directement sur amazone .fr ?
also interesting that 37 people mention private [browsing] in the attachment from comment 11
chofmann: Is this bug about the spike in comments, Zone Alarm, or both? It seems to be both right now, but it's not clear to me that Zone Alarm is the only cause of this spike. This is likely us just getting more comments in general after fixing the throttling bug combined with Zone Alarm being one of our major crashers that causes null signatures... I don't know if that deserves a combined bug, but it might be worth filing the Zone Alarm issue on its own so we can investigate why it causes these types of crashes (and why it crashes at all).
lets make this bug about the increase in zone alarm \N crashes that seem to have spiked around the release of 3.5.5. that is the most interesting problem to be solved right now. the increase in comment volume as a result of comment throttling changes is less interesting now that we know that happened. changed the title. bo from sumo was trying to reproduce the fx3.5.5/zone-alarm problem(s). we should get others doing that as well. and try to find a contact.
Summary: Spike in comments submitted for crash [@ \N ] just after 3.5.5 shipped ; compat problem with zone alarm? → Spike in zone alarm [@ \N ] crashes and crash comments just after 3.5.5 shipped and comment throttling adusted
(In reply to comment #15) > lets make this bug about the increase in zone alarm \N crashes that seem to > have spiked around the release of 3.5.5. I don't think there was an increase in such crashes... I think we just noticed them now because of the comment increase.
https://bug528798.bugzilla.mozilla.org/attachment.cgi?id=412470 suggests there is an increase in \N crashes. maybe due to increase migration/usage of 3.5.x, and maybe due to regression(s) and/or changes in compatibilty problems. right now the only "stacktrace-like" thing we have for these \N signatures that can help us to fix problems are the comments, and the useful comments talk mostly about zone-alarm. your right, the increase/not-increase around 3.5.5 isn't (or may not be) the key issue (although it may help in focusing the testing). we just need to get the zone alarm problem diagnosed and fixed.
Summary: Spike in zone alarm [@ \N ] crashes and crash comments just after 3.5.5 shipped and comment throttling adusted → Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted
We already have <http://support.mozilla.com/en-US/kb/Firefox+crashes+when+loading+certain+pages#ZoneAlarm_ForceField> "ZoneAlarm ForceField may cause Firefox to crash on certain websites. Try clearing the ForceField virtual cache and reboot the computer to see if that resolves the issue. If you need more help, visit ZoneAlarm Support." Is this the same issue? I'll add the "zone alarm" keyword to the article. It didn't turn up in your search, because "ZoneAlarm" is one word in the article. I've filed bug 529347 to cover spaces in search.
ah, might also be that I've searching from a mac and this is a windows problem. maybe user agent sniffing is getting in my way on search and viewing the content? I can see the article listed in comment 18, but I have to view source about the blurb on zone alarm. comment 3 has some suggestions of the terms that users are using to refer to the problem. we might also add zonelabs and zone labs to the search term matching to these articles.
Most of the new comments that we now have visibility into via the crash data don't talk about page loading problems. Here is a list that needs to get investigated and maybe documented. I'll list these problems and user comment one per bugzilla comment problems on upgrade \N 3.5.5 200911051639 ZoneAlarm Extreme Security Forcefield kills Firefox 3.5.5. | Version 3.5.4 worked fine. I'm using ZoneAlarm Extreme Security version 9.0.114.000, ForeField 1.5.36.15. If I | turn off ForceField then the crashing stops. \N 3.5.5 200911061014 I was trying to update ZoneAlarm Extreme Security from 8.0.400.020 to 9.0, however, the system had crashed prior to the reboot moments ago and Firefox was running at the time. \N 3.5.5 200911061140 Running ZoneAlarm Extreme Security Suite w/ ForceField protected browser ver9.0.114.000 \N 3.5.5 200911061518 This is after I loaded ZoneAlarm Extreme Security version 9.1.008.000 looks like similar reports in german and maybe dutch but I don't speak those.
the there is something about the virtualization feature which may or may not be connected to the upgrade problems talked about in comment 20 \N 3.5.5 200911061407 The new Firefox 3.5.5 seems to be incompatible with the virtualization feature of ZoneAlarm Forcefield. Firefox will run occasionally with virtualization enabled, but most of the time I get crashes like this. If I turn off virtualization, Firefox runs normally. Brgds, Jesper Goll \N 3.5.5 200911061635 Crashes when Zonealarm Forcefield Virtualization is turned on \N 3.5.5 200911070119 I'm using ZoneAlarm Extreme Security V9.1.008 with Forcefield Virtualization enabled. I didn't get a crash report when Virtualization was not enabled. Is the crash anyway related to ZA ES Virtualization??? \N 3.5.5 200911081819 I upgraded to Zone Alarm Extreme and it offered Force Field as an option. I just update FF to 3.5.5 and when I enable Virtualization FF crashes. before I updated your latest version of FF it would work. If I turn off Virtualization your browser works for me. Any way a problem with Force Fields Virtualization and 3.5.5 or a bad download of FF update. | low on the scale of computer savy on my part. THX \N 3.5.5 200911092141 I got it to work by shutting down zone alarm virtualization. I use zone alarm extreme. check it out on there website. Everything was fine before the firefox up-grade todoy. \N 3.5.5 200911120815 This started to crash when I switchen on ForceField Virtual Browser (Zone Alarm Security Suite). This is a new problem since the latest Firefox update. | | I'll follow up with Zone Alarm as well. possible similar reports in german dutch and spanish
while comment 20 was mostly about seeing the problem when zone alarm updates there are others that talk about seeing the problem when firefox updates to 3.5.5 \N 3.5.5 200911051639 ZoneAlarm Extreme Security Forcefield kills Firefox 3.5.5. | Version 3.5.4 worked fine. I'm using ZoneAlarm Extreme Security version 9.0.114.000, ForeField 1.5.36.15. If I | turn off ForceField then the crashing stops. \N 3.5.5 200911061219 Is acting weird with zone alarm security with this new update \N 3.5.5 200911061219 Is acting weird with zone alarm security with this new update \N 3.5.5 200911061556 "I have just updated to Firefox 3.5.5 & Zone alarm tells me ""Zone alarm Force Field detected problem with stability of protected browser. It will be restarted."" But it doesn't restart. If I try to restart FireFox the same error appears. FireFox now won't work. HELP :( | I'm using: | ZoneAlarm Extreme Security version:8.0.400.020 | ZoneAlarm ForceField 1.3.153.0"
then there are a group of comments that suggest some connection to private browsing.. \N 3.5.5 200911121011 Zone Alarm Force field Private Browser \N 3.5.5 200911121207 Fire fox 3.5 crashes when opening a private browser using ZoneAlarm Extreme Security Version:8.0.298.035. My previous version of Firefox didn't crash. \N 3.6b2 200911121215 Also this beta (like the 3.5) version crashes when opening a private browser using ZoneAlarm Extreme Security Version:8.0.298.035. See also my report of a couple of minutes ago...Thanks for your help. \N 3.5.5 200911121815 Tried to use the Private browser fuction in Zone Lab Extreme Security. This was after upgrade to 3.5.5 and a cold reboot of my WinXP system. Zlab worked fine with the earlier version
in comment 23, maybe its "firefox private browsing" or maybe its "zone alarm private browsing" or maybe zone alarm private browsing uses the virtuallization feature... all these pieces might fit together to describe one bug e.g. "firefox x is not compatible with zonealarm y with the zonelarm private browser feature in virtuallization mode" or maybe these are all separate bugs.
Taking this to find a contact there. In the meantime, has QA tried to install ZoneAlarm and find ways to make it crash per the comments?
Assignee: nobody → dsicore
Keywords: qawanted
QA will investigate.
(In reply to comment #26) > QA will investigate. will look into this !
I just talked them (will create another bug with contact info), and they are aware of at least one major crash in combination with Fx 3.5.5. I was told they think they fixed it in: ZoneAlarm Extreme Security version 9.0.114.000 which includes another product: ForceField 1.5.36.15. They also ship that product separately. It's the ForceField product that is causing the problems. They said they thought they fixed it in the 1.5.x version, and it shipped two days ago, in an automatic update. We need to keep an eye on these comments to see if we continue to see that version listed. Also, I've asked them to cc themselves on this bug, and I'll ask them for STRs if they have them. In the meantime, we should probably install the latest ZoneAlarm Extreme and turn on the virtualization feature, and see what happens.
Bug description: Latest FireFox update 3.5.5 which was released yesterday is not compatible with ForceField virtualization. FireFox started to use some additional Windows internals, which were not properly emulated by ForceField virtualization. This issue is reproducible only when virtualization is enabled (not a default option for Extreme). There is no workaround for it. The only workaround is temporary disable virtualization. A silent update was delivered beginning 11/13 for ZoneAlarm Extreme users, and is due on 11/20 for ForceField stand-alone users. There are for more Extreme users than there are standalone users.
Hi Jordy, Thanks for the update. We can help to watch the uptake on the updates and confirm it reduces the crashes. It would also be helpful to us if we could understand why this particular crash is causing problems in the collection or transmission or processing of the crash dumps, and the resulting Null Signature we are seeing in this case. Comments on this, or analysis you have run across in your investigations, in how how we might reproduce or view this problem would help us a lot.
Is there a free version or shared license that we can obtain to test out Extreme Security 9.0.114.000? If so, who can i get in touch with to get a licensed copy? Forcefield 1.5.36.15 as well.
I hear we have a licensed copy now. Let's verify the STR and get verification noted here.
Crash confirmed. However i am unable to retrieve my crash stack on about:crashes yet because the browser never loads up in time to get to my profile. Working on getting a stack trace. No crash on 1.9.2 nightly. Repro: 1) Install Fx3.5.5: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 2) install zone alarm Extreme from: http://www.zonealarm.com/security/en-us/anti-virus-spyware-free-download.htm 3) in zone alarm, goto Browser Security > Settings > Advanced > Virtualization > Enable Virtualization (check on). Enable Force Field. 4) Start Fx3.5.5 5) On startup, it crashes immediately before the firefox profile even loads.
Tony: Does the crash reporter come up? We're actually getting some of these crashes submitted (clearly), would be good to confirm the stack (or lack thereof) is the same.
Yes, crash reporter is fired as soon as you try to startup firefox. I've submitted 2 reports already with my email address and comment about zone alarm, so if you can sort crash-stats by that, hopefully you'll retrieve the signature.
(In reply to comment #35) > Yes, crash reporter is fired as soon as you try to startup firefox. I've > submitted 2 reports already with my email address and comment about zone alarm, > so if you can sort crash-stats by that, hopefully you'll retrieve the > signature. crash confirmed working on the stack with a debugger - Tomcat
(In reply to comment #35) > Yes, crash reporter is fired as soon as you try to startup firefox. I've > submitted 2 reports already with my email address and comment about zone alarm, > so if you can sort crash-stats by that, hopefully you'll retrieve the > signature. I don't see any comments in the quick glance of your comments from yesterday useing the search for 'zone alarm' and 'zonealarm' if you disable zone alarm, get firefox launched again, then look at about:crashes can you get a crash report id?
Yeah... it's much easier to see it from about:crashes than for us to try and scour crash reports for your reports...
> crash confirmed working on the stack with a debugger > > - Tomcat (148.da8): Access violation - code c0000005 (!!! second chance !!!) eax=00000000 ebx=109db954 ecx=00000000 edx=0000000e esi=0161b2e0 edi=0161b2e0 eip=101c4946 esp=0012fc24 ebp=0012fc88 iopl=0 nv up ei pl zr na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 > 110: mMainThread->GetPRThread(&mMainPRThread); xul!nsThreadManager::Init+0x8e: 101c4946 8b0481 mov eax,dword ptr [ecx+eax*4] ds:0023:00000000=???????? 0:000> kp ChildEBP RetAddr 0012fc2c 100e2aa2 xul!nsThreadManager::Init(void)+0x8e [e:\builds\moz2_slave\win32_build\build\xpcom\threads\nsthreadmanager.cpp @ 110] 0012fc60 011b9cdf xul!NS_InitXPCOM3_P(class nsIServiceManager ** result = 0x10244489, class nsIFile * binDirectory = 0x0f4a7de8, class nsIDirectoryServiceProvider * appFileLocationProvider = 0x0fc78500, struct nsStaticModuleInfo * staticComponents = 0x0003b385, unsigned int componentCount = 0x3f706800)+0x73 [e:\builds\moz2_slave\win32_build\build\xpcom\build\nsxpcominit.cpp @ 555] 0012fc6c 1006344c MOZCRT19!free(void * ptr = 0x108adc7c)+0x1f [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\jemalloc.c @ 6404] 0012fc88 101b5413 xul!nsLocalFile::Release(void)+0xac [e:\builds\moz2_slave\win32_build\build\xpcom\io\nslocalfilewin.cpp @ 763] 0012fdcc 011b8d3a xul!ScopedXPCOMStartup::Initialize(void)+0x22 [e:\builds\moz2_slave\win32_build\build\toolkit\xre\nsapprunner.cpp @ 1013] 0012fe64 10060685 MOZCRT19!arena_run_dalloc(struct arena_s * arena = 0x1036cc5e, struct arena_run_s * run = 0x1037c2e2, int dirty = 269313376)+0x1ca [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\jemalloc.c @ 3733] 0012ff40 00401213 xul!nsLocalFile::GetParent(class nsIFile ** aParent = 0x00000000)+0x525 [e:\builds\moz2_slave\win32_build\build\xpcom\io\nslocalfilewin.cpp @ 2253] 0012ff80 00401432 firefox!wmain(int argc = 17481, wchar_t ** argv = 0x79706f43)+0x213 [e:\builds\moz2_slave\win32_build\build\toolkit\xre\nswindowswmain.cpp @ 110] 0012ffc0 7c817077 firefox!__tmainCRTStartup(void)+0x152 [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\crtexe.c @ 591] 0012fff0 00000000 kernel32!BaseProcessStart+0x23
Exception Analysis FAULTING_IP: xul!nsThreadManager::Init+8e [e:\builds\moz2_slave\win32_build\build\xpcom\threads\nsthreadmanager.cpp @ 110] 101c4946 8b0481 mov eax,dword ptr [ecx+eax*4] EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff) ExceptionAddress: 101c4946 (xul!nsThreadManager::Init+0x0000008e) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 00000000 Parameter[1]: 00000000 Attempt to read from address 00000000 FAULTING_THREAD: 00000da8 DEFAULT_BUCKET_ID: NULL_POINTER_READ PROCESS_NAME: firefox.exe ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s". EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s". EXCEPTION_PARAMETER1: 00000000 EXCEPTION_PARAMETER2: 00000000 READ_ADDRESS: 00000000 FOLLOWUP_IP: xul!nsThreadManager::Init+8e [e:\builds\moz2_slave\win32_build\build\xpcom\threads\nsthreadmanager.cpp @ 110] 101c4946 8b0481 mov eax,dword ptr [ecx+eax*4] NTGLOBALFLAG: 70 APPLICATION_VERIFIER_FLAGS: 0 PRIMARY_PROBLEM_CLASS: NULL_POINTER_READ BUGCHECK_STR: APPLICATION_FAULT_NULL_POINTER_READ LAST_CONTROL_TRANSFER: from 100e2aa2 to 101c4946 STACK_TEXT: 0012fc2c 100e2aa2 109db954 00000000 0012fcec xul!nsThreadManager::Init+0x8e [e:\builds\moz2_slave\win32_build\build\xpcom\threads\nsthreadmanager.cpp @ 110] 0012fc60 011b9cdf 0160025c 0164d180 1006344c xul!NS_InitXPCOM3_P+0x73 [e:\builds\moz2_slave\win32_build\build\xpcom\build\nsxpcominit.cpp @ 555] 0012fc6c 1006344c 01631460 01631460 00000000 MOZCRT19!free+0x1f [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\jemalloc.c @ 6404] 0012fc88 101b5413 0012fcec 016311c0 0012fd60 xul!nsLocalFile::Release+0xac [e:\builds\moz2_slave\win32_build\build\xpcom\io\nslocalfilewin.cpp @ 763] 0012fdcc 011b8d3a 00000000 21343f58 014b0040 xul!ScopedXPCOMStartup::Initialize+0x22 [e:\builds\moz2_slave\win32_build\build\toolkit\xre\nsapprunner.cpp @ 1013] 0012fe64 10060685 016311c0 0012ff40 01631160 MOZCRT19!arena_run_dalloc+0x1ca [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\jemalloc.c @ 3733] 0012ff40 00401213 00000001 0161e090 01613200 xul!nsLocalFile::GetParent+0x525 [e:\builds\moz2_slave\win32_build\build\xpcom\io\nslocalfilewin.cpp @ 2253] 0012ff80 00401432 00000001 0162e080 01612790 firefox!wmain+0x213 [e:\builds\moz2_slave\win32_build\build\toolkit\xre\nswindowswmain.cpp @ 110] 0012ffc0 7c817077 0166f6f2 0166f77e 7ffdb000 firefox!__tmainCRTStartup+0x152 [e:\builds\moz2_slave\win32_build\build\obj-firefox\memory\jemalloc\src\crtexe.c @ 591] 0012fff0 00000000 004015b0 00000000 78746341 kernel32!BaseProcessStart+0x23 FAULTING_SOURCE_CODE: 106: } 107: 108: // We need to keep a pointer to the current thread, so we can satisfy 109: // GetIsMainThread calls that occur post-Shutdown. > 110: mMainThread->GetPRThread(&mMainPRThread); 111: 112: #ifdef XP_WIN 113: TlsSetValue(gTLSIsMainThreadIndex, (void*) 1); 114: #elif defined(NS_TLS) 115: gTLSIsMainThread = true; SYMBOL_STACK_INDEX: 0 SYMBOL_NAME: xul!nsThreadManager::Init+8e FOLLOWUP_NAME: MachineOwner MODULE_NAME: xul IMAGE_NAME: xul.dll DEBUG_FLR_IMAGE_TIMESTAMP: 4aef7f2a STACK_COMMAND: ~0s ; kb FAILURE_BUCKET_ID: NULL_POINTER_READ_c0000005_xul.dll!nsThreadManager::Init BUCKET_ID: APPLICATION_FAULT_NULL_POINTER_READ_xul!nsThreadManager::Init+8e WATSON_IBUCKET: 1557706059 WATSON_IBUCKETTABLE: 1 WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/firefox_exe/1_9_1_3593/4aef8082/xul_dll/1_9_1_3593/4aef7f2a/c0000005/001c4946.htm?Retriage=1 Followup: MachineOwner ---------
Update: For the ZoneAlarm Extreme product where ForceField is built in: As noted, the user must turn virtualization on (it's off by default) and be on XP OS. We suspect the large number of crashes are from repeat attempts by the same individual rather than all unique users - are you able to confirm? These ZA Extreme users have received an update to address this issue. However the update only works for the latest Extreme version which not all users have switched to yet. And the update only takes affect after a computer reboot. Unfortunately we haven't been telling people about this reboot. We will start to tell users this, I don't have a date yet. This will reduce crash reports. Meanwhile the standalone ForceField version will fix this issue when the new version goes out this Friday - this will reduce crash reports further. Thanks for all your work on this.
(In reply to comment #39) > Yeah... it's much easier to see it from about:crashes than for us to try and > scour crash reports for your reports... unfortunately in this case about:crashes is empty. The Report was indicated as "sent successfully" from the crashreporter, but there is no report listed under about:crashes and only the InstallTime20091102152451 file in the crash reports folder.
(In reply to comment #42) > For the ZoneAlarm Extreme product where ForceField is built in: As noted, the > user must turn virtualization on (it's off by default) and be on XP OS. We > suspect the large number of crashes are from repeat attempts by the same > individual rather than all unique users - are you able to confirm? We don't track information about individual users (like a unique identifier), so there's no way for us to confirm that.
(In reply to comment #43) > (In reply to comment #39) > > Yeah... it's much easier to see it from about:crashes than for us to try and > > scour crash reports for your reports... > > unfortunately in this case about:crashes is empty. The Report was indicated as > "sent successfully" from the crashreporter, but there is no report listed under > about:crashes and only the InstallTime20091102152451 file in the crash reports > folder. yep, i was comment the same. i had tried restarting firefox with enabled virtualization off, and about:crashes doesnt show anything. I left a few comments just now saying "zone alarm, zonealarm, tchung" can you search on that?
(In reply to comment #45) > (In reply to comment #43) > > (In reply to comment #39) > > > Yeah... it's much easier to see it from about:crashes than for us to try and > > > scour crash reports for your reports... > > > > unfortunately in this case about:crashes is empty. The Report was indicated as > > "sent successfully" from the crashreporter, but there is no report listed under > > about:crashes and only the InstallTime20091102152451 file in the crash reports > > folder. > > yep, i was comment the same. i had tried restarting firefox with enabled > virtualization off, and about:crashes doesnt show anything. > > I left a few comments just now saying "zone alarm, zonealarm, tchung" can you > search on that? I'll be able to search for that in the daily crash data report that I get after midnight. If about:crashes doesn't show anything, I'm guessing that I won't see anything. the corruption that might be happening with this bug might in some cases might be: 1) getting in the way with breakpad sending the report, 2) socorro being able to process the report -e.g. gives \N signatures, 3) actually having some crashes with a stack and more data than just comments. but this is just a theory.
re: comment 42. we aren't really able to detect the number of unique users that might be hitting the crash since we limit the amount of user identifiable information that is sent in the reports. we can say that we are processing about 130-150 reports per day that might be related to this crash. that number is a sampling of total crash volume and might be up to 15 times higher for actual counts. looking at comments its a combination of the same user repeatedly crashing maybe once or twice, and other users that are hitting the crash for the first time as they update or install firefox 3.5.5 or the current zone alarm. the virtualization and private browsing feature might be more popular than you suspect ;-) it will be great to see patch go live and we can track the decline in crashes when that happens. we could also put up information on support.mozilla.org and link to your site with more details on how to solve the problem.
(In reply to comment #46) > I'll be able to search for that in the daily crash data report that I get after > midnight. Chris, In case this data helps your search at all. Details of specific crashes i created. BuildID: 20091102152451 CrashTime: 1258665633 Email: tchung@mozilla.com InstallTime: 1258593506 ProductName: Firefox SecondsSinceLastCrash: 6 StartupTime: 1258665633 Throttleable: 1 URL: Vendor: Mozilla Version: 3.5.5 --------------- BuildID: 20091102152451 CrashTime: 1258665774 Email: tchung@mozilla.com InstallTime: 1258593506 ProductName: Firefox SecondsSinceLastCrash: 4 StartupTime: 1258665774 Throttleable: 1 URL: Vendor: Mozilla Version: 3.5.5
We created new update package that asks people to reboot. We will QA it on Monday and will push it to production update server. It will replace current update package for FireFox 3.5.5 fix. This should start to reduce crashes. The ForceField standalone update will go live on MOnday (was originally today).
(In reply to comment #16) > (In reply to comment #15) > > lets make this bug about the increase in zone alarm \N crashes that seem to > > have spiked around the release of 3.5.5. > > I don't think there was an increase in such crashes... I think we just noticed > them now because of the comment increase. I went back and did a bit of analysis on this. https://bug530418.bugzilla.mozilla.org/attachment.cgi?id=413935 shows we were effective throttling to about 5 comments per minute up until 11/05. then when the socorro system was changed the overall average comment/minute rate might have doubled, and the peak comment/minute rate might have gone up by 4x. It's good that we have comment throttling fixed. It's good that we have the zone alarm problem diagnosed and fixes in the works. Once that fix goes live we look at other possible comments in \N signature list for crash the we might be able to diagnose by looking at the comments and spin off other bug(s). Fore those that are interested bug 530418 is on file now about getting more eyeballs on the stream of incoming comments so we can spot problems like this and act faster when they appear.
for Zone Alarm Force Field is still see the Crash on Startup. Not fixed in this version : ZoneAlarm Force Field: 1.3.153.0 - also checked for updates and its saying its the latest version
The total numbers of no-signature crashes (all releases) for the last week and a half or so are: 20091117 5885 20091118 5823 20091119 5587 20091120 5579 20091121 5459 20091122 5753 20091123 5837 20091124 5628 20091125 5924 20091126 5628 20091127 5628 20091128 5530 20091129 6152
(In reply to comment #52) > for Zone Alarm Force Field is still see the Crash on Startup. Not fixed in this > version : > > ZoneAlarm Force Field: 1.3.153.0 - also checked for updates and its saying its > the latest version in Zone Alarm Extreme Security this Problem is fixed with latest Updates ! ZoneAlarm Extreme Security Version i tested: 9.1.008.000 with included ZoneAlarm ForceField 1.5.53.4
1.5.053.049 is the version of the ForceField standalone that has this fix. Previous versions will crash with Firefox. We should be moving all ForceField users to this new version - I'm checking to make sure this is happening. 9.1.008 of ZoneAlarm Extreme is the version that has the fix.
I confirm no crashes anymore with the zone alarm update. Running: - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 - Forcefield 1.5.53.4
(In reply to comment #55) > 1.5.053.049 is the version of the ForceField standalone that has this fix. > Previous versions will crash with Firefox. We should be moving all ForceField > users to this new version - I'm checking to make sure this is happening. > > 9.1.008 of ZoneAlarm Extreme is the version that has the fix. Hi Jordy, thanks for the great work and help on fixing this Problem ! In the stand alone version i got a kind of "check for update completed and you have the latest version" pop-up with the Version Number 1.3.153.0 - shall i check this again ?
Summary: Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted → Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted [@ (signature unavailable)]
This crash now (also?) appears with a signature of @0x101c4946. It's currently #34 in the topcrash list for 3.5.5.
Summary: Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted [@ (signature unavailable)] → Spike in zone alarm [@ \N ] crashes in crash comments just after 3.5.5 shipped and comment throttling adusted [@ (signature unavailable)] [@ @0x101c4946]
Are you seeing crash reports go down now? You should see a visible drop now that we've updated our user base to a 3.5-compatible version.
looks like we saw a drop of about 1000-1500 crashes per day with \N signature around dec 2.
Keywords: qawanted
Severity: normal → S3

I think there is not much actionable here, if any of those crashes is still a thing we'll have newer reports on crash-stats.

Status: NEW → RESOLVED
Closed: 6 months ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: