1112336 - Add memory reporters for the profiler

Status: RESOLVED DUPLICATE of bug 816598

(Core :: Gecko Profiler, defect)

Description

[Nicholas Nethercote [inactive]]

In bug 1109397 Terrence provided evidence that the profiler extension was causing GiBs of "heap-unclassified" memory to accumulate. It would be good to have memory reporters for this. There might also be some kind of leak occurring.

Comment 1

[Nathan Froyd [:froydnj]]

The only significant stacks I see are things like:

  76 blocks in heap block record 1 of 12,863
  136,247,296 bytes (136,235,520 requested / 11,776 slop)
  Individual block sizes: 100,663,296; 12,582,912; 3,145,728; 1,572,864 x 5; 786,432 x 6; 393,216 x 9; 196,608 x 7; 98,304 x 16; 49,152 x 10; 24,576 x 7; 12,288 x 6; 8,192 x 5; 4,096; 2,048
  10.67% of the heap (10.67% cumulative)
  41.55% of unreported (41.55% cumulative)
  Allocated at {
    #01: replace_malloc (/home/froydnj/firefox/libdmd.so)
    #02: malloc (/home/froydnj/firefox/plugin-container)
    #03: moz_xmalloc (/home/froydnj/firefox/plugin-container)
    #04: void std::vector<lul::RuleSet, std::allocator<lul::RuleSet> >::_M_emplace_back_aux<lul::RuleSet const&>(lul::RuleSet const&) (/home/froydnj/firefox/libxul.so)
    #05: lul::SecMap::AddRuleSet(lul::RuleSet const*) (/home/froydnj/firefox/libxul.so)
    #06: lul::Summariser::Rule(unsigned long, int, lul::LExprHow, short, long) (/home/froydnj/firefox/libxul.so)
    #07: lul::DwarfCFIToModule::ValOffsetRule(unsigned long long, int, int, long) (/home/froydnj/firefox/libxul.so)
    #08: lul::CallFrameInfo::ValOffsetRule::Handle(lul::CallFrameInfo::Handler*, unsigned long long, int) const (/home/froydnj/firefox/libxul.so)
    #09: lul::CallFrameInfo::State::DoDefCFAOffset(long) (/home/froydnj/firefox/libxul.so)
    #10: lul::CallFrameInfo::State::DoInstruction() (/home/froydnj/firefox/libxul.so)
    #11: lul::CallFrameInfo::State::InterpretFDE(lul::CallFrameInfo::FDE const&) (/home/froydnj/firefox/libxul.so)
    #12: lul::CallFrameInfo::Start() (/home/froydnj/firefox/libxul.so)
    #13: bool (anonymous namespace)::LoadDwarfCFI<lul::ElfClass64>(std::string const&, lul::ElfClass64::Ehdr const*, char const*, lul::ElfClass64::Shdr const*, bool, lul::ElfClass64::Shdr const*, lul::ElfClass64::Shdr const*, bool, lul::SecMap*, unsigned long, lul::UniqueStringUniverse*, void (*)(char const*)) (/home/froydnj/firefox/libxul.so)
    #14: bool (anonymous namespace)::LoadSymbols<lul::ElfClass64>(std::string const&, bool, lul::ElfClass64::Ehdr const*, bool, (anonymous namespace)::LoadSymbolsInfo<lul::ElfClass64>*, lul::SecMap*, void*, unsigned long, lul::UniqueStringUniverse*, void (*)(char const*)) (/home/froydnj/firefox/libxul.so)
    #15: bool (anonymous namespace)::ReadSymbolDataElfClass<lul::ElfClass64>(lul::ElfClass64::Ehdr const*, std::string const&, std::vector<std::string, std::allocator<std::string> > const&, lul::SecMap*, void*, unsigned long, lul::UniqueStringUniverse*, void (*)(char const*)) (/home/froydnj/firefox/libxul.so)
    #16: lul::ReadSymbolDataInternal(unsigned char const*, std::string const&, std::vector<std::string, std::allocator<std::string> > const&, lul::SecMap*, void*, unsigned long, lul::UniqueStringUniverse*, void (*)(char const*)) (/home/froydnj/firefox/libxul.so)
    #17: lul::LUL::NotifyAfterMap(unsigned long, unsigned long, char const*, void const*) (/home/froydnj/firefox/libxul.so)
    #18: read_procmaps(lul::LUL*) (/home/froydnj/firefox/libxul.so)
    #19: sLUL_initialization_routine() (/home/froydnj/firefox/libxul.so)
    #20: Sampler::Start() (/home/froydnj/firefox/libxul.so)
    #21: mozilla_sampler_start(int, double, char const**, unsigned int, char const**, unsigned int) (/home/froydnj/firefox/libxul.so)
    #22: mozilla::dom::ContentChild::RecvStartProfiler(mozilla::ProfilerInitParams const&) (/home/froydnj/firefox/libxul.so)
    #23: mozilla::dom::PContentChild::OnMessageReceived(IPC::Message const&) (/home/froydnj/firefox/libxul.so)
    #24: mozilla::ipc::MessageChannel::DispatchAsyncMessage(IPC::Message const&) (/home/froydnj/firefox/libxul.so)
  }

which is presumably the unwind information for libxul, along with other similar, smaller stacks for other libraries.  It'd be good to report this, but it also means converting a lot of the unwinder to use Gecko data structures, rather than std:: ones

I haven't noticed anything specific to the profiler itself showing up, though.

Comment 2

[Julian Seward [:jseward]]

Note, in terms of space use for the unwind info, we could do very
much better than we actually do.  Proposal here:

https://blog.mozilla.org/jseward/2013/09/03/how-compactly-can-cfiexidx-stack-unwinding-info-be-represented/