Closed Bug 1014584 Opened 10 years ago Closed 10 years ago

LockedImageSurface::AllocateBuffer's buffers aren't being reported

Categories

(Core :: Graphics: ImageLib, defect)

Product:
Core
Component:
Graphics: ImageLib
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: froydnj, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [MemShrink:P2]) 

From a Windows DMD session (Ted can provide details if he likes, since he's the one who captured it:

Unreported: 41 blocks in stack trace record 1 of 2,261
 67,174,400 bytes (67,148,160 requested / 26,240 slop)
 27.86% of the heap (27.86% cumulative);  46.51% of unreported (46.51% cumulative)
 Allocated at
   replace_posix_memalign (d:\build\opt-mozilla-central\dist\include\replace_malloc.h:120) 0x70ec2255
   posix_memalign_impl (d:\build\mozilla-central\memory\build\replace_malloc.c:161) 0x70d94902
   mozilla::VolatileBuffer::Init (d:\build\mozilla-central\memory\mozalloc\volatilebufferwindows.cpp:65) 0x70ed127c
   LockedImageSurface::AllocateBuffer (d:\build\mozilla-central\image\src\imgframe.cpp:75) 0x5989bff7
   imgFrame::Init (d:\build\mozilla-central\image\src\imgframe.cpp:231) 0x5989c3f6
   mozilla::image::RasterImage::InternalAddFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1204) 0x59884f6b
   mozilla::image::RasterImage::EnsureFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1348) 0x598854c8
   mozilla::image::RasterImage::EnsureFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1406) 0x59885729
   mozilla::image::Decoder::AllocateFrame (d:\build\mozilla-central\image\src\decoder.cpp:219) 0x5988e38e
   mozilla::image::RasterImage::InitDecoder (d:\build\mozilla-central\image\src\rasterimage.cpp:2118) 0x59886fea
   mozilla::image::RasterImage::RequestDecodeCore (d:\build\mozilla-central\image\src\rasterimage.cpp:2378) 0x59887b54
   mozilla::image::RasterImage::RequestDecode (d:\build\mozilla-central\image\src\rasterimage.cpp:2268) 0x598875f6
   mozilla::image::RasterImage::RequestDecodeIfNeeded (d:\build\mozilla-central\image\src\rasterimage.cpp:3067) 0x5988a323
   mozilla::image::RasterImage::FinishedSomeDecoding (d:\build\mozilla-central\image\src\rasterimage.cpp:3171) 0x5988a795
   mozilla::image::RasterImage::DecodePool::DecodeUntilSizeAvailable (d:\build\mozilla-central\image\src\rasterimage.cpp:3454) 0x5988b5eb
   mozilla::image::RasterImage::DoImageDataComplete (d:\build\mozilla-central\image\src\rasterimage.cpp:1749) 0x59886198
   mozilla::image::RasterImage::OnImageDataComplete (d:\build\mozilla-central\image\src\rasterimage.cpp:1798) 0x598864c1
   imgRequest::OnStopRequest (d:\build\mozilla-central\image\src\imgrequest.cpp:675) 0x5987b2d0
   ProxyListener::OnStopRequest (d:\build\mozilla-central\image\src\imgloader.cpp:2198) 0x598768e2
   nsBaseChannel::OnStopRequest (d:\build\mozilla-central\netwerk\base\src\nsbasechannel.cpp:733) 0x59134c03
   nsInputStreamPump::OnStateStop (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:704) 0x5914239a
   nsInputStreamPump::OnInputStreamReady (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:438) 0x59141b66
   nsOutputStreamReadyEvent::Run (d:\build\mozilla-central\xpcom\io\nsstreamutils.cpp:90) 0x590b824a
   nsThread::ProcessNextEvent (d:\build\mozilla-central\xpcom\threads\nsthread.cpp:715) 0x590cb9cf
 
Unreported: 22 blocks in stack trace record 2 of 2,261
 36,044,800 bytes (36,030,720 requested / 14,080 slop)
 14.95% of the heap (42.81% cumulative);  24.96% of unreported (71.46% cumulative)
 Allocated at
   replace_posix_memalign (d:\build\opt-mozilla-central\dist\include\replace_malloc.h:120) 0x70ec2255
   posix_memalign_impl (d:\build\mozilla-central\memory\build\replace_malloc.c:161) 0x70d94902
   mozilla::VolatileBuffer::Init (d:\build\mozilla-central\memory\mozalloc\volatilebufferwindows.cpp:65) 0x70ed127c
   LockedImageSurface::AllocateBuffer (d:\build\mozilla-central\image\src\imgframe.cpp:75) 0x5989bff7
   imgFrame::Init (d:\build\mozilla-central\image\src\imgframe.cpp:231) 0x5989c3f6
   mozilla::image::RasterImage::InternalAddFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1204) 0x59884f6b
   mozilla::image::RasterImage::EnsureFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1348) 0x598854c8
   mozilla::image::RasterImage::EnsureFrame (d:\build\mozilla-central\image\src\rasterimage.cpp:1406) 0x59885729
   mozilla::image::Decoder::AllocateFrame (d:\build\mozilla-central\image\src\decoder.cpp:219) 0x5988e38e
   mozilla::image::RasterImage::InitDecoder (d:\build\mozilla-central\image\src\rasterimage.cpp:2118) 0x59886fea
   mozilla::image::RasterImage::RequestDecodeCore (d:\build\mozilla-central\image\src\rasterimage.cpp:2378) 0x59887b54
   mozilla::image::RasterImage::RequestDecode (d:\build\mozilla-central\image\src\rasterimage.cpp:2268) 0x598875f6
   mozilla::image::RasterImage::RequestDecodeIfNeeded (d:\build\mozilla-central\image\src\rasterimage.cpp:3067) 0x5988a323
   mozilla::image::RasterImage::FinishedSomeDecoding (d:\build\mozilla-central\image\src\rasterimage.cpp:3171) 0x5988a795
   mozilla::image::RasterImage::DecodePool::DecodeUntilSizeAvailable (d:\build\mozilla-central\image\src\rasterimage.cpp:3434) 0x5988b521
   mozilla::image::RasterImage::DoImageDataComplete (d:\build\mozilla-central\image\src\rasterimage.cpp:1749) 0x59886198
   mozilla::image::RasterImage::OnImageDataComplete (d:\build\mozilla-central\image\src\rasterimage.cpp:1798) 0x598864c1
   imgRequest::OnStopRequest (d:\build\mozilla-central\image\src\imgrequest.cpp:675) 0x5987b2d0
   ProxyListener::OnStopRequest (d:\build\mozilla-central\image\src\imgloader.cpp:2198) 0x598768e2
   nsBaseChannel::OnStopRequest (d:\build\mozilla-central\netwerk\base\src\nsbasechannel.cpp:733) 0x59134c03
   nsInputStreamPump::OnStateStop (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:704) 0x5914239a
   nsInputStreamPump::OnInputStreamReady (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:438) 0x59141b66
   nsOutputStreamReadyEvent::Run (d:\build\mozilla-central\xpcom\io\nsstreamutils.cpp:90) 0x590b824a
   nsThread::ProcessNextEvent (d:\build\mozilla-central\xpcom\threads\nsthread.cpp:715) 0x590cb9cf

CC'ing jwatt, as he's been poking around memory reporting for images recently.
The memory usage here seems like a regression, maybe from OMTC?
Can we change the pref for OMTC and re-run this?  Then we'd know.
Possibly related to bug 962670. Although I thought I fixed it not reporting memory in bug 995880.
The DMD report is from a build from 00ef3a7d7aa7, which is a changeset from Saturday, so it's definitely not fixed.
My Nightly was a week out of date, so it wasn't OMTC. Updating to the latest build (which has OMTC enabled) it doesn't seem as bad, but there's still a little bit. I'll update my DMD build and re-test.

It's definitely the Twitter webapp that's provoking the bad behavior, FWIW.
Haven't updated my DMD build yet, but running yesterday's Nightly I still see in about:memory:
├──407.50 MB (55.58%) ── heap-unclassified
In an updated build with OMTC enabled there's still a bunch of heap-unclassified, and it's still image data, but it's allocated differently:

Unreported: 207 blocks in stack trace record 1 of 2,543
 20,254,720 bytes (19,798,975 requested / 455,745 slop)
 9.28% of the heap (9.28% cumulative);  22.52% of unreported (22.52% cumulative)
 Allocated at
   malloc_impl (d:\build\mozilla-central\memory\build\replace_malloc.c:151) 0x70dd46d2
   moz_malloc (d:\build\mozilla-central\memory\mozalloc\mozalloc.cpp:62) 0x70ec164d
   operator new (d:\build\opt-mozilla-central\dist\include\mozilla\mozalloc.h:269) 0x5ca1a88d
   nsTArray_base<nsTArrayFallibleAllocator,nsTArray_CopyWithMemutils>::EnsureCapacity (d:\build\opt-mozilla-central\dist\include\nstarray-inl.h:122) 0x5c1e592b
   nsTArray_Impl<bool,nsTArrayFallibleAllocator>::SetCapacity (d:\build\opt-mozilla-central\dist\include\nstarray.h:1380) 0x5c623a5b
   mozilla::image::RasterImage::SetSourceSizeHint (d:\build\mozilla-central\image\src\rasterimage.cpp:1893) 0x5c9efade
   mozilla::image::ImageFactory::CreateRasterImage (d:\build\mozilla-central\image\src\imagefactory.cpp:225) 0x5c9fb0ee
   mozilla::image::ImageFactory::CreateImage (d:\build\mozilla-central\image\src\imagefactory.cpp:133) 0x5c9fac5c
   imgRequest::OnDataAvailable (d:\build\mozilla-central\image\src\imgrequest.cpp:815) 0x5c9e4e20
   ProxyListener::OnDataAvailable (d:\build\mozilla-central\image\src\imgloader.cpp:2212) 0x5c9dffae
   imgCacheValidator::OnDataAvailable (d:\build\mozilla-central\image\src\imgloader.cpp:2402) 0x5c9e0a6a
   nsBaseChannel::OnDataAvailable (d:\build\mozilla-central\netwerk\base\src\nsbasechannel.cpp:759) 0x5c295860
   nsInputStreamPump::OnStateTransfer (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:593) 0x5c2a2b3a
   nsInputStreamPump::OnInputStreamReady (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:434) 0x5c2a269f
   nsInputStreamReadyEvent::Run (d:\build\mozilla-central\xpcom\io\nsstreamutils.cpp:90) 0x5c2183ea
   nsThread::ProcessNextEvent (d:\build\mozilla-central\xpcom\threads\nsthread.cpp:715) 0x5c22bc4f
   NS_ProcessNextEvent (d:\build\mozilla-central\xpcom\glue\nsthreadutils.cpp:263) 0x5c1c0d6d
   mozilla::ipc::MessagePump::Run (d:\build\mozilla-central\ipc\glue\messagepump.cpp:95) 0x5c4bac82
   MessageLoop::RunInternal (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:230) 0x5c49b8f3
   MessageLoop::RunHandler (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:223) 0x5c49b852
   MessageLoop::Run (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:197) 0x5c49b7bd
   nsBaseAppShell::Run (d:\build\mozilla-central\widget\xpwidgets\nsbaseappshell.cpp:166) 0x5ce41b59
   nsAppShell::Run (d:\build\mozilla-central\widget\windows\nsappshell.cpp:189) 0x5ce09e27
   nsAppStartup::Run (d:\build\mozilla-central\toolkit\components\startup\nsappstartup.cpp:278) 0x5dbf86b1

Unreported: 198 blocks in stack trace record 2 of 2,543
 19,374,080 bytes (18,938,150 requested / 435,930 slop)
 8.88% of the heap (18.17% cumulative);  21.54% of unreported (44.06% cumulative)
 Allocated at
   malloc_impl (d:\build\mozilla-central\memory\build\replace_malloc.c:151) 0x70dd46d2
   moz_malloc (d:\build\mozilla-central\memory\mozalloc\mozalloc.cpp:62) 0x70ec164d
   operator new (d:\build\opt-mozilla-central\dist\include\mozilla\mozalloc.h:269) 0x5ca1a88d
   nsTArray_base<nsTArrayFallibleAllocator,nsTArray_CopyWithMemutils>::EnsureCapacity (d:\build\opt-mozilla-central\dist\include\nstarray-inl.h:122) 0x5c1e592b
   nsTArray_Impl<bool,nsTArrayFallibleAllocator>::SetCapacity (d:\build\opt-mozilla-central\dist\include\nstarray.h:1380) 0x5c623a5b
   mozilla::image::RasterImage::SetSourceSizeHint (d:\build\mozilla-central\image\src\rasterimage.cpp:1893) 0x5c9efade
   mozilla::image::ImageFactory::CreateRasterImage (d:\build\mozilla-central\image\src\imagefactory.cpp:225) 0x5c9fb0ee
   mozilla::image::ImageFactory::CreateImage (d:\build\mozilla-central\image\src\imagefactory.cpp:133) 0x5c9fac5c
   imgRequest::OnDataAvailable (d:\build\mozilla-central\image\src\imgrequest.cpp:815) 0x5c9e4e20
   ProxyListener::OnDataAvailable (d:\build\mozilla-central\image\src\imgloader.cpp:2212) 0x5c9dffae
   nsBaseChannel::OnDataAvailable (d:\build\mozilla-central\netwerk\base\src\nsbasechannel.cpp:759) 0x5c295860
   nsInputStreamPump::OnStateTransfer (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:593) 0x5c2a2b3a
   nsInputStreamPump::OnInputStreamReady (d:\build\mozilla-central\netwerk\base\src\nsinputstreampump.cpp:434) 0x5c2a269f
   nsInputStreamReadyEvent::Run (d:\build\mozilla-central\xpcom\io\nsstreamutils.cpp:90) 0x5c2183ea
   nsThread::ProcessNextEvent (d:\build\mozilla-central\xpcom\threads\nsthread.cpp:715) 0x5c22bc4f
   NS_ProcessNextEvent (d:\build\mozilla-central\xpcom\glue\nsthreadutils.cpp:263) 0x5c1c0d6d
   mozilla::ipc::MessagePump::Run (d:\build\mozilla-central\ipc\glue\messagepump.cpp:95) 0x5c4bac82
   MessageLoop::RunInternal (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:230) 0x5c49b8f3
   MessageLoop::RunHandler (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:223) 0x5c49b852
   MessageLoop::Run (d:\build\mozilla-central\ipc\chromium\src\base\message_loop.cc:197) 0x5c49b7bd
   nsBaseAppShell::Run (d:\build\mozilla-central\widget\xpwidgets\nsbaseappshell.cpp:166) 0x5ce41b59
   nsAppShell::Run (d:\build\mozilla-central\widget\windows\nsappshell.cpp:189) 0x5ce09e27
   nsAppStartup::Run (d:\build\mozilla-central\toolkit\components\startup\nsappstartup.cpp:278) 0x5dbf86b1
   XREMain::XRE_mainRun (d:\build\mozilla-central\toolkit\xre\nsapprunner.cpp:4012) 0x5dbaf123
The allocation stacks from comment 0 and comment 7 seem to be very different. The ones in comment 0 are for decoded image data. The ones in comment 7 are for the source (compressed) image data.

As for the ones in comment 7 they are the mSourceData array, which RasterImage::HeapSizeOfSourceWithComputedFallback has code to include in about:memory, so that must not be working properly somehow.
(In reply to Timothy Nikkel (:tn) from comment #8)
> The allocation stacks from comment 0 and comment 7 seem to be very
> different. The ones in comment 0 are for decoded image data. The ones in
> comment 7 are for the source (compressed) image data.
> 
> As for the ones in comment 7 they are the mSourceData array, which
> RasterImage::HeapSizeOfSourceWithComputedFallback has code to include in
> about:memory, so that must not be working properly somehow.

I noticed, when looking at something slightly different a couple of weeks ago, that RasterImages created by layout (e.g. CSS effects) aren't properly featured in about:memory, because they have no source to be registered with the appropriate image loader.  Comment 7 might be a similar case.
Another complication: DMD only counts a heap block as reported if its measured with a MallocSizeOf function created with MOZ_DEFINE_MALLOC_SIZE_OF or MOZ_DEFINE_MALLOC_SIZE_OF_ON_ALLOC. I think some of the image sizes are instead computed analytically, e.g. just "n += height * width * 4". In which case DMD will say "unreported!" despite there being measurements in about:memory.

In the past I've tried and failed to increase the usage of MallocSizeOf in the image memory reporters, because they are frustratingly complicated, and I think sometimes we don't even know if image memory is on the heap or somewhere else(?) Any improvements on that front would be very welcome.
(In reply to Timothy Nikkel (:tn) from comment #8)
> The allocation stacks from comment 0 and comment 7 seem to be very
> different. The ones in comment 0 are for decoded image data. The ones in
> comment 7 are for the source (compressed) image data.

Comment 0 was from a build pre-OMTC, where comment 7 is post-OMTC, which might explain the difference.
Whiteboard: [MemShrink] → [MemShrink:P2]
I've been having severe slowness on my Windows nightly lately, and it's hard to diagnose with this much heap-unclassified.
Ted, do you still see this?

Bug 1059654 has landed which should mean that there are no more images in heap-unclassified (so we can tell more about what's going on) and bug 1033679 landed which removed a leak of images.
Flags: needinfo?(ted)
I haven't noticed this in a while, but I've also been running with e10s enabled for a while so that probably changes all sorts of things.
Flags: needinfo?(ted)
I guess we'll call it WFM unless proven otherwise.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
It's possible that bug 1065818 may have helped here, too.
You need to log in before you can comment on or make changes to this bug.