515946 - jsd: Add support for memory profiling.

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[John J. Barton]

Wes Garland wrote:
> - That counter that gets incremented when JS_malloc gets called (diffs
> before/after pageload might yield something like peak JS heap usage)

Wes, can you point me to 'that counter'?

There might be several different kinds of memory analysis with different goals, but one in particular seems especially easy to implement and for developers to use: "memory profiling".

Suppose we go in to the js engine at the lines that set the CPU-time profiling values and add corresponding heap-size values. 
http://mxr.mozilla.org/mozilla-central/source/js/jsd/jsd_step.c#198

Then we surface these via properties on jsdIScript just in parallel to the current CPU-time properties:
http://mxr.mozilla.org/mozilla-central/source/js/jsd/idl/jsdIDebuggerService.idl#954

Finally we duplicate the Firebug CPU-time profiler UI code to create a heap-size profile view.
http://code.google.com/p/fbug/source/browse/branches/firebug1.5/content/firebug/profiler.js

In the same way that the CPU-time profiler provides very fast, easy to use "what happened to make my CPU-time so long" we get "what happened to make my heap-size so large". 

Does this make sense?

Comment 1

[John J. Barton]

    Wes, can you point me to 'that counter'?


>I don't think there's an API to get at it, but it's a uint32 stored in >cx->thread->gcMallocBytes.  Note that it goes up on malloc but not down on >free.   You can get what's in cx by including jscntxt.h, although that is a >private header and not part of the JSAPI.

That header is included in 
http://mxr.mozilla.org/mozilla-central/source/js/jsd/jsd_xpc.cpp#42
so we could create an api to it and call that in jsd_step.

Curiously that it only goes up is fine, that more closely matches "time". It does mean that after a GC the profiler will report no allocation. So we need to set user expectations, by eg calling this a "High-Water Profiler", rather than a memory profiler.

Comment 2

[John J. Barton]

Looking at jsd_step, this really looks like a "monkey-see/monkey-do" job. The memory values can be set in the same code blocks as the CPU values. It will make the over all method ~2X longer but that seems to be the style of the code anyway.

Comment 3

[John J. Barton]

I believe this bug fits the student-project criteria.

Comment 4

[Boris Zbarsky [:bzbarsky]]

Have you seen http://almaer.com/blog/browser-memory-tool-prototype ?

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

... which is continued by http://www.toolness.com/wp/?p=709 .

Comment 6

[John J. Barton]

Yes I did see Dion's tool, but not the follow on work by Atul Varma, thanks.

The tool proposed here is *very* different in terms of goals.  Rather than answering the question "what objects are in memory now", the proposed tool here would answer "what functions allocated lots of memory". Consequently the use model will be less like "analyze a snapshot of the heap" and more like "a profiler with extra info".  I believe that some problems will naturally fit one model or the other. 

There does not seem to be much chance for overlap on the code unfortunately. I wonder if the tools could be used together some how. But of course that would be interesting only if we had this one...

Comment 7

[timeless]

Please note that I already have a patch for jsdI which exposes object size, which might or might not be what's wanted here.

People have promised me on many occasions that they'd get reviews done for me so i can land my work.

I'm going to proactively mark this as invalid.

Splitting attention from existing bugs and filing them in the wrong place is totally inappropriate. I should not have to read a mailinglist to find out about changes you want to make to my component.

Don't reopen until you've contacted me. I'm not that hard to reach.

Comment 8

[John J. Barton]

(In reply to comment #7)
> Please note that I already have a patch for jsdI which exposes object size,
> which might or might not be what's wanted here.

Is there a bug number for that, sounds interesting.

> 
> People have promised me on many occasions that they'd get reviews done for me
> so i can land my work.

Sorry I can't review and I don't know what work you are referring to.

> 
> I'm going to proactively mark this as invalid.
> 
> Splitting attention from existing bugs and filing them in the wrong place is
> totally inappropriate. I should not have to read a mailinglist to find out
> about changes you want to make to my component.

Based on the name, the "Javascript Debugging APIs" would be a natural fit. However when I used this in the past I was told not to do so. Plus the description insists on reduced test cases, which I don't have.

If you think it belongs elsewhere that is fine by me.

> 
> Don't reopen until you've contacted me. I'm not that hard to reach.

Ok, I did.

Comment 9

[John J. Barton]

(In reply to comment #7)
> Don't reopen until you've contacted me. I'm not that hard to reach.

As I have not seen a reply in a month, I will assume that whatever issues you had are resolved now and I am resetting the status.

Comment 10

[Josh Matthews [:jdm]]

This doesn't really qualify for a student project any longer, since jsd is going away Real Soon Now. We should probably file a new bug if we'd still like memory profiling in the new debugger API.

Comment 11

[Steve Fink [:sfink] [:s:]]

JSD is dead.