930507 - crash in mozalloc_abort(char const* const) | mozalloc_handle_oom(unsigned int) | moz_xmalloc | nsDisplayBackgroundColor::AllocateGeometry(nsDisplayListBuilder*)

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect)

Description

[Wayne Mery (:wsmwk)]

#8 crash for TB24.0.1
not a topcrash in TB24.0 (at least not in top 60)
does not exist in TB17
not found in betas afaict
windows only (87% win7, 6% win8) no Mac or linux crashes with nsDisplayBackgroundColor::AllocateGeometry in signature

This bug was filed from the Socorro interface and is 
report bp-8ba44542-6494-4979-8ce9-4cb6c2131023. TB24.0.1
=============================================================
0	mozalloc.dll	mozalloc_abort(char const * const)	memory/mozalloc/mozalloc_abort.cpp
1	mozalloc.dll	mozalloc_handle_oom(unsigned int)	memory/mozalloc/mozalloc_oom.cpp
2	mozalloc.dll	moz_xmalloc	memory/mozalloc/mozalloc.cpp
3	xul.dll	nsDisplayBackgroundColor::AllocateGeometry(nsDisplayListBuilder *)	layout/base/nsDisplayList.h
4	xul.dll	mozilla::`anonymous namespace'::ContainerState::ProcessDisplayItems(nsDisplayList const &,unsigned int)	layout/base/FrameLayerBuilder.cpp
5	xul.dll	mozilla::FrameLayerBuilder::BuildContainerLayerFor(nsDisplayListBuilder *,mozilla::layers::LayerManager *,nsIFrame *,nsDisplayItem *,nsDisplayList const &,mozilla::FrameLayerBuilder::ContainerParameters const &,gfx3DMatrix const *,unsigned int)	layout/base/FrameLayerBuilder.cpp
6	xul.dll	nsDisplayList::PaintForFrame(nsDisplayListBuilder *,nsRenderingContext *,nsIFrame *,unsigned int)	layout/base/nsDisplayList.cpp
7	xul.dll	nsDisplayList::PaintRoot(nsDisplayListBuilder *,nsRenderingContext *,unsigned int)	layout/base/nsDisplayList.cpp
8	xul.dll	nsLayoutUtils::PaintFrame(nsRenderingContext *,nsIFrame *,nsRegion const &,unsigned int,unsigned int)	layout/base/nsLayoutUtils.cpp
9	xul.dll	PresShell::Paint(nsView *,nsRegion const &,unsigned int)	layout/base/nsPresShell.cpp
10	xul.dll	nsViewManager::ProcessPendingUpdatesForView(nsView *,bool)	view/src/nsViewManager.cpp
11	xul.dll	nsViewManager::ProcessPendingUpdates()	view/src/nsViewManager.cpp
12	xul.dll	nsRefreshDriver::Tick(__int64,mozilla::TimeStamp)	layout/base/nsRefreshDriver.cpp
13	xul.dll	mozilla::RefreshDriverTimer::TickDriver(nsRefreshDriver *,__int64,mozilla::TimeStamp)	layout/base/nsRefreshDriver.cpp 


bp-6f0a8515-960d-4bd2-9028-8009f2130920 TB24.0

Comment 1

[Philip Chee]

CC some layout people in case they have any insights into this crash.

Comment 2

[Wayne Mery (:wsmwk)]

firefox, no crashes.
Thus far, no useful comments from users

Comment 3

[Matt Woodrow (:mattwoodrow)]

I don't think this is really a layout bug. The allocation that is failing is a rather small one, so it appears that we're actually out of memory.

I think we need to find what's using up most of the memory, which certainly isn't this.

Comment 4

[ISHIKAWA, Chiaki]

(In reply to Matt Woodrow (:mattwoodrow) from comment #3)
> I don't think this is really a layout bug. The allocation that is failing is
> a rather small one, so it appears that we're actually out of memory.
> 
> I think we need to find what's using up most of the memory, which certainly
> isn't this.

I just noticed that I had to update my valgrind/memcheck suppression file because valgrind/memcheck 
spews out new leak warnings ("definite" type) after running valgrind/memcheck 
for the first time in about two weeks (the source from comm-central was refereshed the day before.)
The new suppression entries *may* have a clue to this. But I admit that this is a long shot.

Oh, OK. This bugzilla is about a bug experienced on Windows platform only.

Forget about my comment. My experience is on linux.
But it is true that there seem to be always some leaks, and some may manifest in a worse fashion on
Windows.




TIA

Comment 5

[Wayne Mery (:wsmwk)]

per bug 919986, some users are seeing multiple crash sigs, such as this one.

FWIW, many crashes indicate using only 30-60% of memory.

Comment 6

[Wayne Mery (:wsmwk)]

Another tentative link is bug 913447.   Daniel has both: bp-872c2323-8ca6-4f11-8233-e19582131014 bp-88531635-587a-4aa7-a3f0-575962131024. And in version 17 he was crashing per bug 701533

So, either we have totally new crashes in this, bug 913447, and others. Or a new signature collection that is a continuation of version 17 topcrashes like the never ending bug 701533 triumvirate

Comment 7

[Wayne Mery (:wsmwk)]

another similar collection from User B - Abhay 
- mozalloc_abort(char const* const) | mozalloc_handle_oom(unsigned int) | moz_xmalloc | nsDisplayBackgroundColor::AllocateGeometry(nsDisplayListBuilder*) 
bp-69fe08b4-fbc9-4c90-8ae9-6f7492131028
-  mozalloc_abort(char const* const) | mozalloc_handle_oom(unsigned int) | moz_xmalloc | nsSocketTransport::PostEvent(unsigned int, tag_nsresult, nsISupports*)
bp-7bd712b4-b9a3-463d-9542-5efbd2131024
- mozalloc_abort(char const* const) | NS_DebugBreak_P | AbortIfOffMainThreadIfCheckFast | nsCycleCollector::Forget2(nsPurpleBufferEntry*) bp-757e79f8-3d37-4550-adbd-7a3692130903
(unclear if these fit the model)
-  nsMsgDBFolder::NotifyIntPropertyChanged(nsIAtom*, int, int) 
bp-1f6d6608-78ce-475b-a4ce-9cc1a2130925
-  XPC_WN_NoHelper_Finalize 
9a530bc3-24b2-4d15-b48a-0bae32131024

User A = daniel

Comment 8

[ISHIKAWA, Chiaki]

One question.

Does mozilla's crash information collection tries to
obtain the information about the corruption of code, i.e., if the executable code
is not trashed somehow? MS's error collection facility checks if there is a single bit error
in the loaded DLL and if so, throws out the data, as it regards such
error "hardware (memory) error". 

When someone's machine is susceptible to frequent crashes, I tend to
suspect hardware: chipsets and the lifetime of components. That is why I am using a board with ECC memory (yeah, expensive, but added comfort is woth it.)

I am asking this because:
In the collected information, I noticed that there is an adaptor vendor and adaptor device ID.
I was counting these if there is a pattern in the reports. Poor man's data mining.
I started this because I thought it refers to graphics adaptor. 
However, it looks it is more likely be a network interface since in one report I noticed it is tied to cisco vpn driver (!?).

I think it is better to collect graphics board information since
 - often times, the latest driver from the graphics chip vendor is to blame for graphics corruption,
   blanking of screen, or even outright crashes, 
 - often times, the particular bug is caused only for one type of graphics chip and
   may not affect the other chips.

Surely there are network-related problems, but it is relatively easy to spot network issues in contrast to random crashes caused by misbehaving graphics driver. I don't know why, but I have this impression.

So at least, on windows where the majority of the users are, we should collect grpahics chip information, I think. Should I file a bug on this if Adaptor information is not of graphics chip?
(If it is, why Cisco VPN was mentioned for one entry is a mystery to me.)

TIA


TIA

Comment 9

[Wayne Mery (:wsmwk)]

(In reply to ISHIKAWA, Chiaki from comment #8)
>...
> I am asking this because:
> In the collected information, I noticed that there is an adaptor vendor and
> adaptor device ID.
> I was counting these if there is a pattern in the reports. Poor man's data
> mining.
> I started this because I thought it refers to graphics adaptor. 
> However, it looks it is more likely be a network interface since in one
> report I noticed it is tied to cisco vpn driver (!?).
> ...
> Surely there are network-related problems, but it is relatively easy to spot
> network issues in contrast to random crashes caused by misbehaving graphics
> driver. I don't know why, but I have this impression.
> 
> So at least, on windows where the majority of the users are, we should
> collect grpahics chip information, I think. Should I file a bug on this if
> Adaptor information is not of graphics chip?
> (If it is, why Cisco VPN was mentioned for one entry is a mystery to me.)
> 
> TIA
which crash(s) is the cisco VPN?

perhaps Ted can help for some of your other questions.

Comment 10

[(not currently active) Ted Mielczarek]

(In reply to ISHIKAWA, Chiaki from comment #8)
> One question.
> 
> Does mozilla's crash information collection tries to
> obtain the information about the corruption of code, i.e., if the executable
> code
> is not trashed somehow? MS's error collection facility checks if there is a
> single bit error
> in the loaded DLL and if so, throws out the data, as it regards such
> error "hardware (memory) error". 

No, we don't have any such instrumentation right now. We do include the 256 bytes of memory around the faulting instruction pointer, so it would be possible to compare those vs. the stock binaries (although with relocations etc that might be tricky to get right).

> I am asking this because:
> In the collected information, I noticed that there is an adaptor vendor and
> adaptor device ID.
> I was counting these if there is a pattern in the reports. Poor man's data
> mining.

If a report against this data would be useful to you you can file a bug under Socorro: Data Request outlining what you need.

> I started this because I thought it refers to graphics adaptor. 
> However, it looks it is more likely be a network interface since in one
> report I noticed it is tied to cisco vpn driver (!?).

It is the graphics adapter:
http://mxr.mozilla.org/mozilla-central/source/widget/windows/GfxInfo.cpp#728

> I think it is better to collect graphics board information since
>  - often times, the latest driver from the graphics chip vendor is to blame
> for graphics corruption,
>    blanking of screen, or even outright crashes, 
>  - often times, the particular bug is caused only for one type of graphics
> chip and
>    may not affect the other chips.

There's a specific check for Cisco VPN:
http://mxr.mozilla.org/mozilla-central/source/widget/windows/GfxInfo.cpp#700

This is from bug 626994, where it was causing a set of crashes.

Comment 11

[ISHIKAWA, Chiaki]

Hi,

Sorry I missed this post earlier. (Got tied up with day job.)

As Ted explained it so clearly some crashes showed CiscoVPN software driver information as a driver for adaptor in the crash info, and I was
confused. From Ted's explanation I understand that CiscoVPN software caused problems in the past and
so was specifically detected and printed.

I can't recall now which is which, but a few crash reports I checked in October DID show such
CiscoVPN fingerprints, but maybe they are not directly related to this bug.

My main confusion was that where the graphics adaptor is usually reported, CiscoVPN driver was
reported, and that turned out to be intentional. So it is OK.

OTOH,
http://mxr.mozilla.org/mozilla-central/source/widget/windows/GfxInfo.cpp#695

695 #if defined(MOZ_CRASHREPORTER)
696 /* Cisco's VPN software can cause corruption of the floating point state.
697  * Make a note of this in our crash reports so that some weird crashes
698  * make more sense */
699 static void
700 CheckForCiscoVPN() {
701   LONG result;
702   HKEY key;
703   /* This will give false positives, but hopefully no false negatives */
704   result = RegOpenKeyExW(HKEY_LOCAL_MACHINE, L"Software\\Cisco Systems\\VPN Client", 0, KEY_QUERY_VALUE, &key);
705   if (result == ERROR_SUCCESS) {
706     RegCloseKey(key);
707     CrashReporter::AppendAppNotesToCrashReport(NS_LITERAL_CSTRING("Cisco VPN\n"));
708   }
709 }
710 #endif
711 

Changing the string on 707 to something like "Cisco VPN (may screw up FPU status)\n" 
might help the debuggers in the future to understand why Cisco VPN driver is put in the spot usually
reserved for graphics driver.

TIA

Comment 12

[Wayne Mery (:wsmwk)]

Way more Seamonkey crashes than Thunderbird. SM crashes with reporter's email address:
bp-1ff760e0-7b78-46c8-9ea7-7a3e32140113
bp-d8ffc006-4d76-4e4e-b541-f61e62140114
bp-6c3e343d-9052-4ebe-956f-e718d2140115
bp-53a4c421-61e8-4754-ab93-dae6e2140115
bp-d481e6e6-89ce-490f-8554-a28122140113
bp-edcbdb74-7dc8-44cb-98b6-b58402140116
bp-baceb7fd-6494-44e7-838c-9bf2a2140115
We can attempt to contact them if someone things we need more information.

crashes whose users say they are not using vpn:
- bp-6fedcc4c-e8a2-4bb6-8d5e-0ba252131209 Ken, Thunderbird "i was typing a group mail to nine people i have sent to before when it just went, message wasn't saved to drafts"
- bp-1da55241-e9fa-4434-bfca-cd16a2131029 Paul, Thunderbird "I did a deep scan (IOBit Advanced System Care pro) on my desktop, followed by a full registry and disk deep defrag (using the latest Auslogics software) - since then (fingers crossed), no TBird crashes."


Note, several users with this crash sig also have many others crash signatures, which suggests antivirus software or the like may be involved in some cases.


Several users only experience the crash once. like bp-906d1e3d-600c-430f-9ca2-e08102131028 ben

Comment 13

[Wayne Mery (:wsmwk)]

nothing on crash-stats after 2014-06-16 calendar date, for any version of Windows.
Fixed by a blocklist?