Closed Bug 813214 Opened 12 years ago Closed 11 years ago

Add memory profiling to SPS

Categories

(Core :: Gecko Profiler, defect)

x86
macOS
defect
Not set
normal

Tracking

()

RESOLVED WONTFIX

People

(Reporter: BenWa, Unassigned)

References

Details

The goal is to create a memory tool that can at the minimum collect the allocation/free location, block size and time in a custom/nightly tools. The aim is to supplement functionality of more heavy weight tools like Valgrind/DMD that can't be deployed to users. The tool should be perform enough to run as an always-on add-on to monitor memory usage in day-to-day browsing.

To perform fast enough to be used day-to-day the tool will likely have to only analyze a subset of memory allocations. How this block is selected is still up for discussion. It should be tunable to 100% to convert this tool into an absolute memory tracer at the cost of performance.

Here are the use case we can solve:
1) Show which areas of the code base allocate the most memory. This can help us see if component are churning memory but is otherwise not very helpful on its own.
2) Show how the memory has changed between 2 points in time. Say we open-close a tab + Full GC. Any piece allocated during the trace and retained after was not released after closing the tab. We can then investigate if we should release that memory earlier.
3) Look for about:memory dark matter if we instrument the memory reporters. A partial DMD.
4) Use memprotect+timer to look for soft-leaks. This is a more advanced, experimental, idea.
5) Dump the data to JSON and let people write python script to do their own analysis.
6) Focus on the allocation of specific objects.
> The aim is to supplement functionality of more heavy weight tools like Valgrind/DMD that can't be 
> deployed to users.

The overhead of non-Valgrind-based newdmd is equal to the overhead of recording memory backtraces.  That is, we don't need to choose between this and newdmd.

> The tool should be perform enough to run as an always-on add-on to monitor memory usage in 
> day-to-day browsing.

While this would be great, I'm not convinced it's possible to reduce the overhead to an acceptable level.  malloc() is very hot, plus we incur extra memory usage from storing these backtraces.

Unlike with the CPU profiler, there's no limit to how long we have to store this metadata for, which also adds to the overhead.

In other words, while this is a nice goal, I'd probably be happy with a tool which didn't meet this criterion.

In general, I'm most interested in DMD-like functionality, to supplement about:memory.  The other use cases above are interesting,  but not something I've felt a need for.
Glandium blogged about a useful way to replace the memory allocator:
http://glandium.org/blog/?p=2848
(In reply to Justin Lebar [:jlebar] from comment #1)

Alright I see no point in working on something that would replicate newdmd. I am however still interested in exploring providing feature 1) 2) 4) in a nightly build.
Depends on: replace-malloc
Is this going anywhere?  about:memory can do diffs now, which covers point 2).
Nope.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
fwiw, I'd be interested in being able to populate Profiler objects with memory profiles as well (or make them available somewhere else). We're discussing this feature in bug 933026.

Feel free to reopen if you think this is the appropriate place to do that or we can use Panos' memory profiler (see bug 923275).
You need to log in before you can comment on or make changes to this bug.