Open Bug 1241710 Opened 8 years ago Updated 2 years ago

Large number of allocations unaccounted in about:memory related to safebrowsing (detected by DMD in heap-unclassified)

Categories

(Toolkit :: Safe Browsing, defect, P3)

defect

Tracking

()

People

(Reporter: jujjyl, Unassigned)

References

Details

Attachments

(1 file)

Running Firefox in torture test mode, _without_ private browsing enabled, on emunittest 0.5 mode for about an hour, and closing all tabs after that, releasing all memory with about:memory, and capturing a memory log with DMD, the top unaccounted for memory block that remains is the following

Unreported {
  ~679 blocks in heap block record 1 of 1,837
  ~2,779,147 bytes (~2,779,147 requested / ~0 slop)
  Individual block sizes: ~4,093 x 679
  4.39% of the heap (4.39% cumulative)
  12.75% of unreported (12.75% cumulative)
  Allocated at {
    #01: realloc_impl (d:\gecko-dev\memory\build\replace_malloc.c:193)
    #02: moz_xrealloc (d:\gecko-dev\memory\mozalloc\mozalloc.cpp:105)
    #03: sk_realloc_throw (d:\gecko-dev\gfx\skia\skia\src\ports\skmemory_mozalloc.cpp:31)
    #04: nsTArray_base<nsTArrayInfallibleAllocator,nsTArray_CopyWithMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator> (d:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nstarray-inl.h:183)
    #05: nsTArray_Impl<enum nsGridContainerFrame::TrackSize::StateBits,nsTArrayInfallibleAllocator>::AppendElement<enum nsGridContainerFrame::TrackSize::StateBits,nsTArrayInfallibleAllocator> (d:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nstarray.h:1591)
    #06: nsUrlClassifierPrefixSet::MakePrefixSet (d:\gecko-dev\toolkit\components\url-classifier\nsurlclassifierprefixset.cpp:122)
    #07: nsUrlClassifierPrefixSet::SetPrefixes (d:\gecko-dev\toolkit\components\url-classifier\nsurlclassifierprefixset.cpp:81)
    #08: mozilla::safebrowsing::LookupCache::ConstructPrefixSet (d:\gecko-dev\toolkit\components\url-classifier\lookupcache.cpp:580)
    #09: mozilla::safebrowsing::LookupCache::Build (d:\gecko-dev\toolkit\components\url-classifier\lookupcache.cpp:165)
    #10: mozilla::safebrowsing::Classifier::ApplyTableUpdates (d:\gecko-dev\toolkit\components\url-classifier\classifier.cpp:668)
    #11: mozilla::safebrowsing::Classifier::ApplyUpdates (d:\gecko-dev\toolkit\components\url-classifier\classifier.cpp:311)
    #12: nsUrlClassifierDBServiceWorker::ApplyUpdate (d:\gecko-dev\toolkit\components\url-classifier\nsurlclassifierdbservice.cpp:587)
    #13: nsUrlClassifierDBServiceWorker::FinishUpdate (d:\gecko-dev\toolkit\components\url-classifier\nsurlclassifierdbservice.cpp:552)
    #14: nsRunnableMethodArguments<>::apply<mozilla::WatchManager<mozilla::dom::HTMLMediaElement>::PerCallbackWatcher,void (__cdecl mozilla::WatchManager<mozilla::dom::HTMLMediaElement>::PerCallbackWatcher::*)(void) __ptr64> (d:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nsthreadutils.h:664)
    #15: nsRunnableMethodImpl<void (__cdecl mozilla::dom::cache::Context::ThreadsafeHandle::*)(void) __ptr64,1>::Run (d:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nsthreadutils.h:872)
    #16: nsThread::ProcessNextEvent (d:\gecko-dev\xpcom\threads\nsthread.cpp:989)
    #17: NS_ProcessNextEvent (d:\gecko-dev\xpcom\glue\nsthreadutils.cpp:297)
    #18: mozilla::ipc::MessagePumpForNonMainThreads::Run (d:\gecko-dev\ipc\glue\messagepump.cpp:356)
    #19: MessageLoop::RunInternal (d:\gecko-dev\ipc\chromium\src\base\message_loop.cc:235)
    #20: MessageLoop::RunHandler (d:\gecko-dev\ipc\chromium\src\base\message_loop.cc:228)
    #21: MessageLoop::Run (d:\gecko-dev\ipc\chromium\src\base\message_loop.cc:202)
    #22: nsThread::ThreadFunc (d:\gecko-dev\xpcom\threads\nsthread.cpp:403)
    #23: _PR_NativeRunThread (d:\gecko-dev\nsprpub\pr\src\threads\combined\pruthr.c:419)
    #24: pr_root (d:\gecko-dev\nsprpub\pr\src\md\windows\w95thred.c:91)
  }
}

Some thoughts:

a) since this is a relatively large source of memory allocations, it would be good to get accounted for in about:memory
b) can this be a memory leak? Is this memory ever freed?
Component: Private Browsing → Safe Browsing
Product: Firefox → Toolkit
Summary: Large number of allocations unaccounted in about:memory related to private browsing (detected by DMD in heap-unclassified) → Large number of allocations unaccounted in about:memory related to safebrowsing (detected by DMD in heap-unclassified)
Thanks, I was confused when reporting on privatebrowsing vs safebrowsing, but naturally these are two different features.

In most of the runs I do with DMD, even when I don't visit more than two pages, this callstack shows up in the logs of untracked allocations at one of the highest spots, so it would be good to get this hoisted away from heap-unclassified in about:memory.
Priority: -- → P5
Priority: P5 → P3
Whiteboard: #sbv4-m8
Profiling a new WebAssembly demo page in DMD today, visit http://mzl.la/webassemblydemo for the test case.

Having only that page open in current FF Nightly on Windows, the main process in about:memory shows

├───22.98 MB (13.22%) ── heap-unclassified

and the second highest individual source of heap-unclassified is

Unreported {
  3 blocks in heap block record 3 of 8,111
  593,920 bytes (593,920 requested / 0 slop)
  Individual block sizes: 524,288; 65,536; 4,096
  0.74% of the heap (9.78% cumulative)
  3.00% of unreported (39.56% cumulative)
  Allocated at {
    #01: realloc_impl (e:\gecko-dev\memory\build\replace_malloc.c:193)
    #02: moz_xrealloc (e:\gecko-dev\memory\mozalloc\mozalloc.cpp:106)
    #03: nsTArray_base<nsTArrayInfallibleAllocator,nsTArray_CopyWithMemutils>::EnsureCapacity<nsTArrayInfallibleAllocator> (e:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nstarray-inl.h:183)
    #04: nsTArray_Impl<mozilla::OggDemuxer::SeekRange,nsTArrayInfallibleAllocator>::AppendElement<mozilla::OggDemuxer::SeekRange,nsTArrayInfallibleAllocator> (e:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nstarray.h:2132)
    #05: mozilla::safebrowsing::LookupCacheV2::ReadCompletions (e:\gecko-dev\toolkit\components\url-classifier\lookupcache.cpp:528)
    #06: mozilla::safebrowsing::LookupCacheV2::Open (e:\gecko-dev\toolkit\components\url-classifier\lookupcache.cpp:410)
    #07: mozilla::safebrowsing::Classifier::GetLookupCache (e:\gecko-dev\toolkit\components\url-classifier\classifier.cpp:1350)
    #08: mozilla::safebrowsing::Classifier::CopyInUseLookupCacheForUpdate (e:\gecko-dev\toolkit\components\url-classifier\classifier.cpp:1014)
    #09: mozilla::safebrowsing::Classifier::ApplyUpdatesBackground (e:\gecko-dev\toolkit\components\url-classifier\classifier.cpp:707)
    #10: nsUrlClassifierDBServiceWorker::ApplyUpdatesBackground (e:\gecko-dev\toolkit\components\url-classifier\nsurlclassifierdbservice.cpp:754)
    #11: mozilla::detail::RunnableFunction<<lambda_a486a2b8106ee48eb89ec3ccdaa3b955> >::Run (e:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\nsthreadutils.h:314)
    #12: mozilla::SyncRunnable::Run (e:\gecko-dev\obj-x86_64-pc-mingw32\dist\include\mozilla\syncrunnable.h:112)
    #13: nsThread::ProcessNextEvent (e:\gecko-dev\xpcom\threads\nsthread.cpp:1270)
    #14: NS_ProcessNextEvent (e:\gecko-dev\xpcom\threads\nsthreadutils.cpp:389)
    #15: mozilla::ipc::MessagePumpForNonMainThreads::Run (e:\gecko-dev\ipc\glue\messagepump.cpp:338)
    #16: MessageLoop::RunHandler (e:\gecko-dev\ipc\chromium\src\base\message_loop.cc:232)
    #17: MessageLoop::Run (e:\gecko-dev\ipc\chromium\src\base\message_loop.cc:212)
    #18: nsThread::ThreadFunc (e:\gecko-dev\xpcom\threads\nsthread.cpp:502)
    #19: _PR_NativeRunThread (e:\gecko-dev\nsprpub\pr\src\threads\combined\pruthr.c:419)
    #20: pr_root (e:\gecko-dev\nsprpub\pr\src\md\windows\w95thred.c:96)
    #21: o__realloc_base[C:\WINDOWS\System32\ucrtbase.dll +0x1cab0]
    #22: BaseThreadInitThunk[C:\WINDOWS\System32\KERNEL32.DLL +0x8364]
    #23: RtlUserThreadStart[C:\WINDOWS\SYSTEM32\ntdll.dll +0x670d1]
  }
}
Assignee: nobody → jujjyl
Status: NEW → ASSIGNED
Whiteboard: #sbv4-m8 → #sbv4-m9
No longer blocks: safebrowsingv4
Whiteboard: #sbv4-m9

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.

Assignee: jujjyl → nobody
Status: ASSIGNED → NEW
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: