628152 - mozalloc_abort (possibly NS_RUNTIMEABORT) crashes in gfxFontGroup::BuildFontList [@ mozalloc_abort(char const* const) ][@ mozalloc_abort(char const* const) | mozcrt19.dll@0x1327f ] (was [@ mozcrt19.dll@0x1327f ])

Status: VERIFIED FIXED

(Core :: Graphics, defect)

Description

[Scoobidiver (away)]

It is a new crash signature in the trunk that includes various stack traces.
It is #5 top crashes in 4.0b10pre for the last week.
This bug is when frame 2 is equal to gfxFontGroup::BuildFontList

There are interesting comments about this crash:
"Firefox was minimized when crash happen. I was working with FontExpert 2010. It was the first sesion after restart by update and downthemall was downloading a file (600mb) from fileserve."
"backed twice from gtalk dl page to google.com"

Signature	mozcrt19.dll@0x1327f
UUID	eab9a051-6b5b-4c24-a899-e10502110123
Time 	2011-01-23 04:15:26.434011
Uptime	939
Install Age	4523 seconds (1.3 hours) since version was first installed.
Product	Firefox
Version	4.0b10pre
Build ID	20110122030336
Branch	2.0
OS	Windows NT
OS Version	6.1.7600
CPU	x86
CPU Info	AuthenticAMD family 16 model 10 stepping 0
Crash Reason	EXCEPTION_BREAKPOINT
Crash Address	0x734e1a39
App Notes 	AdapterVendorID: 1002, AdapterDeviceID: 6739, AdapterDriverVersion: 8.801.0.0

Frame 	Module 	Signature [Expand] 	Source
0 	mozalloc.dll 	mozalloc_abort 	memory/mozalloc/mozalloc_abort.cpp:77
1 	mozcrt19.dll 	mozcrt19.dll@0x1327f 	
2 	xul.dll 	gfxFontGroup::BuildFontList 	
3 	xul.dll 	gfxFontGroup::gfxFontGroup 	gfx/thebes/gfxFont.cpp:1970
4 	xul.dll 	gfxWindowsPlatform::CreateFontGroup 	gfx/thebes/gfxWindowsPlatform.cpp:535
5 	xul.dll 	nsThebesFontMetricsConstructor 	gfx/src/thebes/nsThebesGfxFactory.cpp:52
6 	xul.dll 	mozilla::GenericFactory::CreateInstance 	obj-firefox/xpcom/build/GenericFactory.cpp:48
7 	xul.dll 	nsCOMPtr_base::~nsCOMPtr_base 	obj-firefox/dist/include/nsAutoPtr.h:969
8 	xul.dll 	nsFontCache::GetMetricsFor 	gfx/src/thebes/nsThebesDeviceContext.cpp:181
9 	xul.dll 	nsCOMPtr_base::assign_from_gs_contractid 	obj-firefox/xpcom/build/nsCOMPtr.cpp:132
10 	xul.dll 	nsBlockReflowState::nsBlockReflowState 	layout/generic/nsBlockReflowState.cpp:147

More reports at:
http://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=exact&query=&range_value=2&range_unit=weeks&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=&signature=mozcrt19.dll%400x1327f

Comment 1

[Scoobidiver (away)]

Crashes with this stack trace started in 4.0b10pre/20110121153543.
They are also in 4.0b10.

The regression range is:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=8d52e3b68ca6&tochange=16bd82195df8
There are two possible culprits: bug 624310, bug 602792.

Comment 2

[Scoobidiver (away)]

It is now #1 top crasher in 4.0b10pre and 4.0b10.

Comment 3

[Joe Drew (not getting mail)]

Is this an OOM abort or a corruption abort?

In either case, John, this is a big crasher already on b10.

Comment 4

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

Nothing near this code should be RUNTIMEABORT()ing.  I'd guess OOM abort but the stacks I randomly sampled aren't very helpful.

Comment 5

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

There is a suspicious RUNTIMEABORT in there, my fault.

Comment 6

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

It's sad that we have to speculate about whether or not this was a RUNTIMEABORT().  I filed bug 628885 and posted a patch.  Hopefully that can get into nightlies soon.

Comment 7

[chris hofmann]

Correlation to startup or time of session315 total crashes for mozcrt19.dll@0x1327f on 20110124-crashdata.csv
46 startup crashes inside 30 sec.
79 startup crashes inside 3 min.
44 repeated crashes inside 3 min. of last crash

urls for testing
  37 
  25 jar:file:///C:/Program%20Files/Mozilla%20Firefox%204.0%20Beta%207/omni.jar!/chrome/browser/content/browser/aboutHome.xhtml
  20 \N
   5 about:blank
   2 jar:jar:file:///C:/Users/Tolga/AppData/Roaming/Mozilla/Firefox/Profiles/m8f0b3o1.default/extensions/%7B64161300-e22b-11db-8314-0800200c9a66%7D.xpi!/chrome/speeddial.jar!/content/speeddial.xul
   2 http://www.minitorneos.com/juegos/juego.php?juego=21&sala=2
   2 http://www.jizzonline.com/videos/brazilian-2187985.html
   2 http://www.google.fr/ig?refresh=1
   2 http://vkontakte.ru/feed
   2 http://sunonline.webzen.com/News/Event/Default.aspx?iBS=1855
   2 http://source.android.com/
   2 http://softget.net/tags/OneTouch/page/3
   2 http://int.ask.com/hpstatic
   2 http://fotomolduras.com/moldura-foto_1547-tudo-florido.htm
   2 http://clickjogos.uol.com.br/jogos-3d/

also these domains show up..

  37 //
  27 jar:file://
  20 \N//
   5 jar:jar:file://
   5 http://www.google.com
   5 http://www.facebook.com
   5 about:blank//
   4 http://www.orkut.com.br
   4 http://vkontakte.ru
   3 http://www.youtube.com
   3 http://www.free-lance.ru
   3 http://int.ask.com

Comment 8

[Benjamin Smedberg]

You don't have to speculate: load the minidump(s) into MSVC and get a better stacktrace.

Comment 9

[Sheila Mooney]

This is showing up high on the list for early Beta10 crashes.

Comment 10

[John Daggett (:jtd)]

Steps to reproduce:

1. Start browser
2. Add a font to the font folder
3. Open a new window

Result: browser aborts

Adding a font to the font folder causes a WM_FONTCHANGE message to be received.  The message handling code for this message calls through to InitFontList.  The lazy init logic doesn't reset mInitialized within InitFontList so the code to lazily construct the font family list never gets called.  This leaves an empty font list and causes a runtime about when looking up a font family name.

I'm somewhat puzzled why so many users would be seeing this, since users aren't typically adding fonts to the font folder.  I suspect it may be related to plugins dynamically loading fonts (e.g. some versions of the JVM do this).

Comment 11

[John Daggett (:jtd)]

Attachment: patch, fix crash in BuildFontList

Should fix the spate of related dwrite font crashers.  Needs more testing.

Comment 12

[John Daggett (:jtd)]

Attachment: patch, fix crash in BuildFontList

Properly initialize mInitialized to false when rebuilding the font list.  Also, backed out parts of the lazy metric initialization until I can figure out why this appears to be failing in some cases.  Guessing that it's got something to do with invalid fonts but I need to test that more and come up with clear steps to reproduce.

Comment 13

[Jonathan Kew [:jfkthame]]

Comment on attachment 507378 [details] [diff] [review]
patch, fix crash in BuildFontList

>--- a/gfx/thebes/gfxDWriteFontList.cpp
>+++ b/gfx/thebes/gfxDWriteFontList.cpp
>@@ -645,6 +645,8 @@
> {
>     LOGREGISTRY(L"InitFontList start");
> 
>+    mInitialized = PR_FALSE;

Yup, that looks like it could account for a lot of trouble!

While you're here, I think if it'd be worth adding an

    if (!mInitialized) {
        mInitialized = PR_TRUE;
        DelayedInitFontList();
    }

block to gfxDWriteFontList::LookupLocalFont, too. It's cheap, and it seems vaguely conceivable that if the font list gets reset (uninitialized) at odd times, there might be a scenario where it happens that the next font-list operation we try to do is a src:local() lookup.

Comment 14

[chris hofmann]

This is signature is going a bit crazy on the  first full day of beta 10 in release.  3291 crashes yesterday which makes it 10x higher than the #2 rank.

We need to move this up to blocking beta 11. and we need to get beta 11 out on a timely schedule.

Comment 15

[chris hofmann]

I should add that this is probably one of several fixes that we need to knock down volume on this signature.  see also bug 626768 and bug 625709 and probably more that we need to get on file.  working on getting a better breakdown on how much impact this particular fix mnight help in reducing the overall volume.

Comment 16

[chris hofmann]

http://people.mozilla.org/crash_stacks/stack-summary-4.0b11pre.txt shows a sample distribution from the trunk has font group in a high percent of the stacks for this signature.  also running same kind of analysis for beta 10 to see if we get the same kind of distribution there.

Comment 17

[Sheila Mooney]

I second chofmann's comments. We need to get this in ASAP. Bug 626768 is not nominated for 2.0 and 625709 is currently resolved incomplete.

Comment 18

[chris hofmann]

the sample from beta 10 also shows about 75% of mozcrt19.dll@0x1327f have 
gfxFontGroup::BuildFontList() on the stack.

http://people.mozilla.org/crash_stacks/stack-summary-4.0b10.txt

Comment 19

[chris hofmann]

here are some urls with user comments to try and develop some test cases that build confidence in the fix.


http://www.eenadu.net/font.zip I was on a local newspaper site eenadu.net. This website has been visited million times on other versions of FF without crash.

http://www.hulu.com/stand_alone/aaf01e881d5c3a0b66a2d4921a37c838?lcname=ESMABiFhZFNlbGVjdG9yT3B0aW9uBgEGJ2RlZmF1bHRDYXB0aW9uU3R5bGUGAzEGDXVzZXJJRASE9/PCBh1kZWZhdWx0UXVhbGl0eQQBBhV1c2VyUGxhbklEBA4GE2NvbnRlbnRJRASL9/eLBi1zZWxlY3RvclZpZGVvQ29tcGxldGVkAgYXcG9 watching an episode of heroes on hulu

http://community.spiceworks.com/a?uuid=ec852aa0-4c63-012d-d08d-002481f9c91e&&_h=1293&_v=5.0.60009&jsr=r using spiceworks

http://artrack.co.cc/flipside/wp-admin/options-general.php?page=share-and-follow.php It just crashed

\N Suddently stop...Crashed.
\N The mozilla beta has several glitches.

http://slashdot.org/pollBooth.pl Clicked on a web poll on slashdot.org

http://help.e01.divinsa.net/help.htm "Launching firefox from an application ""Divinsa"".  Firefox was already open."

http://www.google.com/search?q=watch+how+i+met+your+mother+online&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:hu:official&client=firefox-a facebook.com-on status update-t akartam írni, az első két bepötyögött betű után fail. nem tudom, segít-e ez bármi.

http://www.bbc.co.uk/news/ theregister, bbc news

http://code.google.com/p/php-calendar/wiki/InstallDoc c r a s h

http://static.ak.fbcdn.net/connect/xd_proxy.php#cb=f2a25a9f9427374&origin=http%3A%2F%2Fminiclip.sapo.pt%2Ffbb0cd093ebcc6&relation=parent.parent&transport=postmessage&type=resize&height=20&width=100 Crash do Firefox
 ananızı sikim

https://www.facebook.com/home.php my browser just did an atomatical update :)

\N seit der beta 10 stürzt er nurnoch ab...

http://s16.de.ikariam.com/index.php?view=city&id=XXXX Datenverlust wegen des Absturzes - sehr ärgerlich

http://dap-ikariam.xobor.de/msg.php?board=539127&Pedit=1&forum=25&Thread=191&msg=861&prev=0&next=112 stürzt dauernd bei der gleichen Seite ab

http://www.haveeru.com.mv/ dhivehi language fonts are not displaying

http://en.wikipedia.org/wiki/Special:Search?search=pulley&sourceid=Mozilla-search Second time today Firefox has crashed. Maybe it's not a FF problem, but one with my computer?

http://forum.paticik.com/read.php?43,5218631,5355225#msg-5355225 facebook'ta profilleri gezerken firefox çöküyor.

https://ntepo:8443/core/j_security_check I had the site for our McAfee ePO server open.

http://www.google.com.br/ do nada o mozilla pediu para fechar, notei que isso é comum com as versões beta.

http://veckonytt.se/arkiv/vn1.pdf Was going to open a new tab, just clicked the + sign and it crashed :( my fox died! :)

https://secure.shared.live.com/_D/F$Live.SiteContent.Messenger/4.2.59120/LocalStorage.html#domain=by155w.bay155.mail.live.com&loaderPath=https://secure.wlxrs.com/rahv5DXVjJzhJWo92gJ4Kg/loader.cxp.js&channelNames=DefaultLocalStorageChannel;MessengerStorage se cuelga seguido no permite continuar con los trabajos. gracias

http://www.shutterstock.com/cat.mhtml?lang=en&search_source=search_form&similar_photo_id=&searchterm=Dollar+Sign+Symbol&anyorall=all&search_cat=&people_gender=&people_age=&people_ethnicity=&people_number=&search_group=all&orient=all&photographer_name=&sea Laptop computer went to sleep with browser open. I was logged into a website. On return, the laptop woke up fine, but when I started browser the site that I was logged onto, the browser crash.

http://support.kaspersky.vn/ fix it

\N 4th or 5th time in the last hour. Maybe I should give up on Firefox.

http://www.urbanfonts.com/fonts/graffiti-fonts.htm 2nd time this happened, i don't know why. Could it have anything to do with me installing a font? That's what it I did the first time too.

http://www.google.com/search?q=free+fonts&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a Crashing when using google search bar  b10 only.

Comment 20

[chris hofmann]

Attachment: crash volume for mozcrt19.dll@0x1327f jan 10-26

looks like this signature spiked on 2011 01 13 with a large volume of crashes from build 4.0b10pre--2011-01-13-03a, but those probably mostly have to do  with the fixes tracked in bug 625709 which was duped from Bug 625709 which recorded that first spike.

between Jan 16-21 we appeared to be in recovery from the first spike period, then on 2011-01-22 we spiked again to 148 crashes with most (111) coming from 4.0b10pre-build-2011-01-21-15

jdagget, are there changes in font code around Jan 21 that would fully explain the second spike or should we keep looking for other possible problems that would explain all of this spike.

Comment 21

[John Daggett (:jtd)]

(In reply to comment #20)
> jdagget, are there changes in font code around Jan 21 that would fully explain
> the second spike or should we keep looking for other possible problems that
> would explain all of this spike.

Yes, the changes that landed for bug 602792 on 1/21 caused the crashes with BuildFontList in the stack:

https://bugzilla.mozilla.org/show_bug.cgi?id=602792#c100

Comment 22

[John Daggett (:jtd)]

Landed on trunk
http://hg.mozilla.org/mozilla-central/rev/3b4f201929d4

Comment 24

[chris hofmann]

(In reply to comment #18)
> the sample from beta 10 also shows about 75% of mozcrt19.dll@0x1327f have 
> gfxFontGroup::BuildFontList() on the stack.
> 
> http://people.mozilla.org/crash_stacks/stack-summary-4.0b10.txt


I took a larger sample of 100 reports from yesterday on b10 and the +75% number is holding up.

Comment 25

[Ben]

> Steps to reproduce:
> 1. Start browser
> 2. Add a font to the font folder
> 3. Open a new window
> Result: browser aborts

Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11 ID:20110203141415

does not crash (4.0b10 crashes with this STR). Seems fixed.

Comment 26

[[:Aleksej]]

Marking verified according to comment #25.