Closed Bug 625753 Opened 14 years ago Closed 14 years ago

AMD64 Firefox 4.0b10pre Crash Reports [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] [@ malloc_rtree_set | arena_run_split ] [@ malloc_rtree_set | KERNELBASE.dll@0x6f3f ]

Categories

(Core :: Memory Allocator, defect)

Product:
Core
Component:
Memory Allocator
Platform:
x86_64
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: chofmann, Assigned: m_kato)

References

Details

(Keywords: crash, meta)

Crash Data

Attachments

(1 file)

fix
1.22 KB, patch
ted
: review+
benjamin
: approval2.0+
Details | Diff | Splinter Review
new high volume regression on trunk. its confusing since none of the sources near the top of the stack have changed in several week or more. stack looks like http://crash-stats.mozilla.com/report/index/df19d6e5-dc2f-4178-a304-964152110113 0 mozcrt19.dll malloc_rtree_set obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2408 1 KERNELBASE.dll KERNELBASE.dll@0x659f 2 xul.dll nsCSSFrameConstructor::DoContentStateChanged layout/base/nsCSSFrameConstructor.cpp:8183 3 mozcrt19.dll chunk_alloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2589 4 mozcrt19.dll huge_malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:4654 5 mozcrt19.dll malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:5873 6 xul.dll gfxImageSurface::gfxImageSurface gfx/thebes/gfxImageSurface.cpp:111 7 mozalloc.dll moz_xmalloc memory/mozalloc/mozalloc.cpp:98 8 mozcrt19.dll malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:5873 9 xul.dll imgFrame::Init modules/libpr0n/src/imgFrame.cpp:227 10 mozcrt19.dll arena_malloc_large obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3820 11 xul.dll mozilla::imagelib::RasterImage::InternalAddFrame modules/libpr0n/src/RasterImage.cpp:771 12 kernel32.dll kernel32.dll@0x4bbe5 13 mozcrt19.dll memset obj-firefox/memory/jemalloc/crtsrc/memset.c:55 14 xul.dll mozilla::imagelib::RasterImage::AppendFrame modules/libpr0n/src/RasterImage.cpp:832 15 xul.dll mozilla::imagelib::nsPNGDecoder::CreateFrame modules/libpr0n/decoders/nsPNGDecoder.cpp:119 16 xul.dll mozilla::imagelib::nsPNGDecoder::info_callback modules/libpr0n/decoders/nsPNGDecoder.cpp:614 17 mozcrt19.dll memcmp obj-firefox/memory/jemalloc/crtsrc/memcmp.c:60 18 xul.dll MOZ_PNG_push_read_chunk modules/libimg/png/pngpread.c:435 19 xul.dll MOZ_PNG_proc_some_data modules/libimg/png/pngpread.c:64 20 xul.dll MOZ_PNG_process_data modules/libimg/png/pngpread.c:41 21 KERNELBASE.dll KERNELBASE.dll@0x1806 22 xul.dll mozilla::imagelib::nsPNGDecoder::WriteInternal modules/libpr0n/decoders/nsPNGDecoder.cpp:349 23 nspr4.dll PR_Now nsprpub/pr/src/md/windows/ntmisc.c:356 24 xul.dll mozilla::imagelib::RasterImage::WriteToDecoder modules/libpr0n/src/RasterImage.cpp:2198 25 xul.dll mozilla::imagelib::imgDecodeWorker::Run modules/libpr0n/src/RasterImage.cpp:2594 26 xul.dll mozilla::imagelib::RasterImage::WriteToRasterImage modules/libpr0n/src/RasterImage.cpp:2671 27 xul.dll nsInputStreamTee::WriteSegmentFun xpcom/io/nsInputStreamTee.cpp:222 28 xul.dll nsPipeInputStream::ReadSegments xpcom/io/nsPipe3.cpp:799 29 mozalloc.dll moz_xmalloc memory/mozalloc/mozalloc.cpp:98 30 xul.dll nsInputStreamTee::ReadSegments xpcom/io/nsInputStreamTee.cpp:275 31 xul.dll xul.dll@0xae72cf 32 xul.dll imgRequest::OnDataAvailable modules/libpr0n/src/imgRequest.cpp:1156 33 xul.dll nsDocLoader::FireOnProgressChange uriloader/base/nsDocLoader.cpp:1272 more reports at http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20KERNELBASE.dll@0x659f one user comment gah.. pages with alot of pictures on them, seem to be killing firefox. could be one or a few users hitting this but the volume stretches out over 3 days now. Jan 12, 2011 10:49 - Jan 14, 2011 07:42 (latest as of bug filing) crash addresses are variable so its not just a high volume of dupes.
test urls 10 \N 8 http://crystalin.dyndns.org:8080/GwtQuake.html 1 http://youtube.aapkaapnatv.com/2011/01/dance-india-dance-doubles-mega-audition_13.html 1 http://yfrog.com/f/h4x0ep/ 1 http://www.service.karlstad.se/vemos2/vemos2_web.dll/lt?lti=XXXXX 1 http://www.sankakucomplex.com/2011/01/13/square-enix-planning-final-fantasy-xiii-2/ 1 http://www.hurtom.com/torrents/forum/viewtopic.php?t=11089 1 http://www.google.be/ 1 http://www.fudzilla.com/ 1 http://www.farmville.com/thankyougift.php?zy_ctoken=null&contentID=XXXX 1 http://www.facebook.com/profile.php?id=XXXXX 1 http://www.facebook.com/pagelet/generic.php/pagelet/home/morestories.php? XXX 1 http://www.facebook.com/ajax/home/feed.php? XXXX 1 http://www.deviantart.com/download/143585075/Candy_by_Mikkoliini.zip 1 http://www.cal-star.com/inventory/details.asp?product=CRODAMOL+SS 1 http://www.boringstories.co.uk/minecraft/output9.png 1 http://uk.wikipedia.org/wiki/%D0%93%D0%BE%D0%BB%D0%BE%D0%B2%D0%BD%D0%B0_%D1%81%D1%82%D0%BE%D1%80%D1%96%D0%BD%D0%BA%D0%B0 1 http://technet.microsoft.com/en-us/library/cc507849.aspx 1 http://sot.wikia.com/wiki/File:Wizards_first_rule_original.jpg 1 http://nvphenm100/SiteScope/accounts/loginDN/htdocs/Reports-0/Report-11_13-01_13_2011.html 1 http://lpf.org.ua/lpftv/2010/?page=2 1 http://listen.grooveshark.com/sidebar.php?ThemeID=4&CurArtist=4522&Gender=F&AgeRange=18-24 1 http://blog.dropbox.com/?p=593 1 http://apps.facebook.com/xd_receiver_v0.4.php#%7 XXX %22mobsters_index 1 http://apps.facebook.com/mobsters-two/?ref_id= XXXX 1 http://appadvice.com/appnn/2011/01/breaking-apple-releases-ios-43-beta-developers/ 1 http://admin.piklio.com/kukadlo.php?server= XXXX 1 http://abc.go.com/watch 1 http://5.52.96.152/~deegee/world-maps/world_20110113-1200130588.png 1 http://10.0.0.100/CastleControl/index.php?&showroom=ALL&shownoti&showat&showpics&showhist&showweath=1&fs20dev=&orderpulldown=&valuetime=&showmenu=1&showroom=ALL
another pile with similar top-of-stack with signature [@ malloc_rtree_set | chunk_alloc ] and the same crash time span. Jan 12, 2011 10:47 - Jan 14, 2011 08:23 and all the same build id 2011 01 12 074539 http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20chunk_alloc http://crash-stats.mozilla.com/report/index/a6a4e91f-e9a2-41fa-9cee-36cd02110114 0 mozcrt19.dll malloc_rtree_set obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2408 1 mozcrt19.dll chunk_alloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2589 2 mozcrt19.dll huge_malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:4654 3 mozjs.dll js::Shape::trace js/src/jsscope.cpp:1466 4 mozcrt19.dll malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:5873 5 mozjs.dll js::gc::MarkChildren js/src/jsgcinlines.h:289 6 xul.dll ChangeTable obj-firefox/xpcom/build/pldhash.c:564 7 xul.dll PL_DHashTableOperate obj-firefox/xpcom/build/pldhash.c:650 8 xul.dll GCGraphBuilder::AddNode xpcom/base/nsCycleCollector.cpp:1429 9 xul.dll GCGraphBuilder::NoteXPCOMChild xpcom/base/nsCycleCollector.cpp:1612 10 xul.dll nsGenericElement::cycleCollection::Traverse content/base/src/nsGenericElement.cpp:4416 11 xul.dll GCGraphBuilder::NoteXPCOMChild xpcom/base/nsCycleCollector.cpp:1615 12 xul.dll nsGenericDOMDataNode::cycleCollection::Traverse content/base/src/nsGenericDOMDataNode.cpp:102 13 xul.dll nsCycleCollector::MarkRoots xpcom/base/nsCycleCollector.cpp:1766 14 xul.dll nsCycleCollector::BeginCollection xpcom/base/nsCycleCollector.cpp:2644 ... ... ...
Summary: Firefox 4.0b10pre Crash Report [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] → Firefox 4.0b10pre Crash Report [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ]
hard to say where this should land. probably not layout, or even jemalloc. could it be garbage collection?
Component: Layout → jemalloc
QA Contact: layout → jemalloc
same top line of stack, and same build, and same crash span with these reports. [@ malloc_rtree_set | arena_run_split ] http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20arena_run_split Frame Module Signature [Expand] Source 0 mozcrt19.dll malloc_rtree_set obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2408 1 mozcrt19.dll arena_run_split obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3042 2 mozcrt19.dll chunk_alloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:2589 3 mozcrt19.dll arena_run_alloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3240 4 mozcrt19.dll arena_malloc_small obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3794 5 mozcrt19.dll arena_malloc_large obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3811 6 mozcrt19.dll malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:5873 7 mozcrt19.dll malloc obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:5873 8 mozcrt19.dll operator new obj-firefox/memory/jemalloc/crtsrc/new.cpp:54 9 mozcrt19.dll arena_malloc_small obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3783 10 mozjs.dll js::detail::HashTable<js::HashMap<unsigned char*,js::LoopProfile*,js::DefaultHasher<unsigned char*>,js::SystemAllocPolicy>::Entry,js::HashMap<unsigned char*,js::LoopProfile*,js::DefaultHasher<unsigned char*>,js::SystemAllocPolicy>::MapHashPolicy,js::SystemAllocPolicy>::init js/src/jshashtable.h:351 11 mozjs.dll js::InitJIT js/src/jstracer.cpp:7644 12 mozjs.dll JSCompartment::init js/src/jscompartment.cpp:112 13 mozcrt19.dll arena_malloc_small obj-firefox/memory/jemalloc/crtsrc/jemalloc.c:3783 14 mozjs.dll js::gc::NewCompartment js/src/jsgc.cpp:2867 15 mozjs.dll JS_NewCompartmentAndGlobalObject js/src/jsapi.cpp:2954 16 xul.dll CreateNewCompartment js/src/xpconnect/src/nsXPConnect.cpp:965 ... ... ...
chofmann, all of the stacks you've listed are completely different. You're going to need to break it down to the stack above the allocator, so e.g. js::detail::Hashtable or js::Shape::trace or gfxImageSurface::gfxImageSurface. It's also possible that the stack here has nothing to do with the crash (the memory corruption happened earlier), in which case we need to figure it out by the regression range. What *is* the regression range?
Component: jemalloc → General
Keywords: meta
QA Contact: jemalloc → general
(In reply to comment #5) > What *is* the regression range? All three of these signature started showing up on Jan 12. First crash times are: Jan 12, 2011 10:47 - malloc_rtree_set | KERNELBASE.dll@0x659f Jan 12, 2011 10:49 - malloc_rtree_set | chunk_alloc Jan 12, 2011 15:21 - malloc_rtree_set | arena_run_split All of the 100+ crashes we have seen across all 3 signatures share one thing in common. They have only been seen on build 2011 01 12 07 4539 So this might have already been fixed some how. I think we should just keep any eye out and if it appears in later builds we can dig deeper. > It's also possible that the stack here has nothing to do with the crash (the > memory corruption happened earlier), in which case we need to figure it out by > the regression range. My guess is this would be the next thing to explore, since a quick scan of of the js::detail::Hashtable or js::Shape::trace or gfxImageSurface::gfxImageSurface wasn't turning up any resent changes.
Note that symbols for Windows x64 builds started working again on Jan 12, see Bug 618385. There have been no new builds for Windows x64 since then.
This may be jemalloc bug for Win64 since crash address isn't 64bit address.
Depends on: 625315
yeah, new batch of crashes yesterday across all three signatures on builds from the 18th and 19th. Otherwise all the crashes continue to be on 2011 01 12 builds. They are also all AMD64 when reported on b10pre, except for the small number of reports that we see on 3.6.x like http://crash-stats.mozilla.com/report/index/7d389842-fb68-4733-acb1-0d57b2110114 malloc_rtree_set...chunk_alloc date total breakdown by build crashes count build, count build, ... 20110119 19 10 4.0b10pre2011011903, 4 4.0b10pre2011011813, 3 4.0b10pre2011011207, 2 4.0b10pre2011011803, ------------------- malloc_rtree_set...arena_run_split date total breakdown by build crashes count build, count build, ... 20110119 9 6 4.0b10pre2011011903, 3 4.0b10pre2011011813, ------------------- malloc_rtree_set...KERNELBASE.dll@0x659f date total breakdown by build crashes count build, count build, ... 20110119 40 16 4.0b10pre2011011903, 14 4.0b10pre2011011813, 10 4.0b10pre2011011207, There are also some comments in the latest batch of reports. http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20KERNELBASE.dll@0x659f > opening many tabs (4 reports) > since the last two updates minefield is crashing a lot, the previous version crashed coz of select and drag n now this one without any reason. > gah.. pages with alot of pictures on them, seem to be killing firefox. http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20chunk_alloc > opening many tabs http://crash-stats.mozilla.com/report/list?signature=malloc_rtree_set%20|%20arena_run_split no comments
Summary: Firefox 4.0b10pre Crash Report [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] → AMD64 Firefox 4.0b10pre Crash Reports [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] [@ malloc_rtree_set | arena_run_split ]
AMD64 is just the name of the CPU architecture, see bug 610828.
Attached patch fix — — Splinter Review
Assignee: nobody → m_kato
Comment on attachment 510197 [details] [diff] [review] fix Although I don't know how to reproduce this, this seems to be Makefile issue. MOZ_MEMORY_SIZEOF_PTR_2POW isn't exported on mozcrt19. If undef, build env detects as 32bit.
Attachment #510197 - Flags: review?(ted.mielczarek)
Comment on attachment 510197 [details] [diff] [review] fix Touching these files always scares the hell out of me, but this looks like a simple change, and it's localized to only x86-64 builds with jemalloc enabled.
Attachment #510197 - Flags: review?(ted.mielczarek) → review+
Comment on attachment 510197 [details] [diff] [review] fix This change is Win64 only.
Attachment #510197 - Flags: approval2.0?
Component: General → jemalloc
QA Contact: general → jemalloc
Hardware: x86 → x86_64
Version: unspecified → Trunk
Summary: AMD64 Firefox 4.0b10pre Crash Reports [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] [@ malloc_rtree_set | arena_run_split ] → AMD64 Firefox 4.0b10pre Crash Reports [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] [@ malloc_rtree_set | arena_run_split ] [@ malloc_rtree_set | KERNELBASE.dll@0x6f3f ]
Attachment #510197 - Flags: approval2.0? → approval2.0+
landed http://hg.mozilla.org/mozilla-central/rev/a74a5384f005 But I keep open status because I want to check crash statistics on next nightly.
no crash from 2010-02-09 nightly.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Crash Signature: [@ malloc_rtree_set | KERNELBASE.dll@0x659f ] [@ malloc_rtree_set | chunk_alloc ] [@ malloc_rtree_set | arena_run_split ] [@ malloc_rtree_set | KERNELBASE.dll@0x6f3f ]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: