If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Add a heap profiler feature to Gecko Profiler

NEW
Unassigned

Status

()

Core
Gecko Profiler
6 years ago
9 months ago

People

(Reporter: BenWa, Unassigned)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
We already have a significant infrastructure for profiling code performance cross-platform (profiling, symbolication, add-on control, front end presentation, js integration soon) that's been designed to be modular and accept optional feature. Currently all feature are performance related and selected in the add-on but it's doesn't have to be that way.

I'd like to suggest that we add heap profiling features. We should add a few hooks to jemalloc on MOZ_PROFILING builds that do a cheap return when the heap profiling is disabled. When it's enabled there's a few things we feature we can provide:

1) Electric fence styles feature to detect out of bounds read/write, usage after free: http://en.wikipedia.org/wiki/Electric_Fence
2) Liveness: We randomly choose to track a block being allocated in jemalloc. Tracked block get no read/write permissions. When they segfault we set them as read/write and record the last use time. Every minute (or the desired resolution) we reset to no read/write on the page giving us a rough estimate within a minute when the page was last used. We can unwind the allocation/last usage point/dealloc. With this info we can find memory we're leaking, or holding on for far too long in caches.

The goal here is not to replace valgrind but to provide some debugging feature that are cross-platform and work with nearly no performance overhead, say < 5%. We can reduce the overheads by profiling subsets of allocations based on a probabilities. Power users with profiling extensions, such as myself, will always be collecting performance and heap data (I already run profiling 24/7 to catch small janks). The benefit is that the data is collected in a realistic browsing usage. Even if it only catches a subset of errors by having people run them all the time we can collect a lot of useful data. Again we're not looking to replace valgrind but provide ways for people to collect data in their regular browsing without any hassle.
Seems like Address Sanitizer already takes care of the first feature above (on Linux and OS X, of course) and more.  Do you think it's useful to have our own version of this functionality as well?
> 1) Electric fence styles feature to detect out of bounds read/write, usage after free: 
> http://en.wikipedia.org/wiki/Electric_Fence

I wouldn't necessarily call this "heap profiling".

In any case, you should look at jemalloc2 [1], which we're actively working to integrate into FF (bug 580408, but most of the action atm is on the jemalloc mailing list).

jemalloc2 has something called a heap profiler, and there's something about red zones in the code.  But I haven't dug into exactly what it does.

[1] http://www.canonware.com/jemalloc/ -- see the dev branch in git; master is old.
(Reporter)

Comment 3

6 years ago
(In reply to Ehsan Akhgari [:ehsan] from comment #1)
> Seems like Address Sanitizer already takes care of the first feature above
> (on Linux and OS X, of course) and more.  Do you think it's useful to have
> our own version of this functionality as well?

The goal I'm aiming for isn't to replace existing tools, but to provide an add-on that randomly audit memory and will catch some errors. If it doesn't get in the way of users people will be willing to run it 24/7.

(In reply to Justin Lebar [:jlebar] from comment #2)
> > 1) Electric fence styles feature to detect out of bounds read/write, usage after free: 
> > http://en.wikipedia.org/wiki/Electric_Fence
> 
> I wouldn't necessarily call this "heap profiling".
> 
> In any case, you should look at jemalloc2 [1], which we're actively working
> to integrate into FF (bug 580408, but most of the action atm is on the
> jemalloc mailing list).
> 
> jemalloc2 has something called a heap profiler, and there's something about
> red zones in the code.  But I haven't dug into exactly what it does.
> 
> [1] http://www.canonware.com/jemalloc/ -- see the dev branch in git; master
> is old.

I'll take a look. I must rather not have to reinvent this code here.

Updated

4 years ago
See Also: → bug 964096
Note that :jimb and I are working on memory tooling for devtools. It is focused on the POV from JS, but will expose everything in the heap.

Follow along with the bugs in this graph: https://bugzilla.mozilla.org/showdependencygraph.cgi?id=960671
(Reporter)

Comment 5

4 years ago
robcee mentioned this to me the other day. I'm going to hold off on this and any similarly features since it should meet our needs.
Blocks: 1329137
You need to log in before you can comment on or make changes to this bug.