72748 - instrument JS

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine, defect, P1)

Description

[Cathleen]

brendan, we need help taking a closer look at JS to see if we can reduce it's 
25% of memory usage allocated by the app.

Comment 1

[Brendan Eich [:brendan]]

Patch coming, more to do after that.  I still think my energy is better spent
fixing some of my 0.9 footprint bugs, but the next patch should help others to
start finding possibly bogus JS object bloat in the chrome.

/be

Comment 2

[Brendan Eich [:brendan]]

Attachment: first round of instrumentation and cleanup (lotsa whitespace changes in nsTraceMalloc.c)

Comment 3

[Brendan Eich [:brendan]]

Attachment: diff -wuN of files in last patch

Comment 4

[Brendan Eich [:brendan]]

sfraser is original contributor of dump_xpc.{cpp,h} -- I'll make sure those
files get a compatible license comment with credits.

/be

Comment 5

[Brendan Eich [:brendan]]

Attachment: page to visit that dumps the GC heap to /tmp/gcdump.out

Comment 6

[Brendan Eich [:brendan]]

Attachment: gzip of a dump done from my dogfood build, which has two browser windows and mail up and running for a day

Comment 7

[Brendan Eich [:brendan]]

Attachment: latest patch (better gcdump format, license comments in dump_xpc.*)

Comment 8

[Brendan Eich [:brendan]]

Attachment: Better html page with form to do controlled dump to named file

Comment 9

[Brendan Eich [:brendan]]

Attachment: latest dump showing revised format (quoted strings, no trailing () on strong ref path)

Comment 10

[Mike Shaver (:shaver emeritus)]

I've got some hacked perl scripts to pull this apart by object type and then
individual object, but I'm not exactly sure what people are looking for.  If we
put the addresses of the objects in the ref path in, and stop truncating it, I
could try to build a set of object graphs and visualize them with webdot or
something.

As it stands now, I can give summary stats from brendan's latest dump:

strings:  7121
doubles:   238
objects: 11190 (!)

Object breakdown by class:

Function                   9483
XULElement                 504
Object                     169
Error                      130
Script                     96
RegExp                     57
NodeList                   27
Text                       23
Array                      23
Node                       22
KeyEvent                   22
Document                   22
With                       21
XPCOutArg                  20
UIEvent                    20
NamedNodeMap               20
JSOptions                  20
InstallVersion             20
InstallTrigger             20
HTMLOptionElement          20
HTMLImageElement           20
HTMLElement                20
Event                      20
Element                    20
CharacterData              20
Attr                       20
15nsXPCComponents          20
Number                     18
String                     17
Math                       16
Date                       15
Call                       14
Boolean                    14
Arguments                  14
7nsJSIID                   9
WindowCollection           8
HTMLDocument               7
Window                     6
HTMLInputElement           5
14nsStringBundle           5
XULDocument                4
XULCommandDispatcher       4
Navigator                  4
HTMLHtmlElement            4
HTMLBodyElement            4
6nsPref                    4
21nsSecureBrowserUIImpl    4
13nsRDFResource            4
13nsCharsetMenu            4
Location                   3
HTMLFormElement            3
14RDFServiceImpl           3
XULTreeElement             2
XStringBundle              2
Q212nsJVMManager8Internal  2
HTMLTextAreaElement        2
HTMLTableSectionElement    2
HTMLTableRowElement        2
HTMLTableElement           2
HTMLTableCellElement       2
HTMLDivElement             2
29nsMsgAccountManagerDataSource 2
23CompositeDataSourceImpl  2
21nsMsgFolderDataSource    2
21nsAutoCompleteResults    2
20RDFXMLDataSourceImpl     2
19nsMsgStatusFeedback      2
19nsMsgComposeService      2
17nsBrowserInstance        2
16nsXULControllers         2
15nsUrlbarHistory          2
15nsCommonDialogs          2
14LocalStoreImpl           2
13nsDragService            2
12nsJVMManager             2
11nsMsgWindow              2
11nsMessenger              2
Q28nsStdURL8Internal       1PrefConfig                 1
nsXULPrototypeScript_compilation_scope 1
HTMLPreElement             1
HTMLParagraphElement       1
HTMLAnchorElement          1
global                     1
chrome:__messenger_content_mailWidgets.xml_mail-emailaddress 1
chrome:__global_content_xulBindings.xml_textfield 1
chrome:__global_content_xulBindings.xml_statusbar-panel 1
chrome:__global_content_xulBindings.xml_progressmeter 1
chrome:__global_content_xulBindings.xml_popups 1
chrome:__global_content_xulBindings.xml_menuitem-iconic 1
chrome:__global_content_xulBindings.xml_menuitem 1
chrome:__global_content_xulBindings.xml_menu-iconic 1
chrome:__global_content_xulBindings.xml_menu 1
chrome:__global_content_xulBindings.xml_iframe 1
chrome:__global_content_xulBindings.xml_browser 1
chrome:__global_content_xulBindings.xml_basetext 1
chrome:__global_content_treeBindings.xml_treeitem 1
chrome:__global_content_treeBindings.xml_tree 1
chrome:__global_content_toolbarBindings.xml_toolbox 1
chrome:__global_content_toolbarBindings.xml_toolbargrippy 1
chrome:__global_content_toolbarBindings.xml_toolbar 1
chrome:__global_content_toolbarBindings.xml_menubar 1
chrome:__global_content_outlinerBindings.xml_outliner 1
chrome:__global_content_outlinerBindings.xml_columnpicker 1
chrome:__global_content_menulistBindings.xml_menubutton-dual-ex 1
chrome:__global_content_autocomplete.xml_autocomplete 1
21RDFContainerUtilsImpl    1
21nsStringBundleService    1
20nsAbAddressCollecter     1
19nsOutlinerBoxObject      1
19nsMsgThreadedDBView      1
19nsMsgAccountManager      1
18nsEditorController       1
18nsBrowserBoxObject       1
17nsMsgHeaderParser        1
16nsMsgMailSession         1
15nsLocaleService          1
15nsFileTransport          1
13nsFileChannel            1

Obviously, more investigation of the Function stuff is required -- how do we
want to break them down?  We could get dump_xpc to tell us source file and
function name, I guess.

(I assume we're running GC before dumping the heap.)

Comment 11

[Brendan Eich [:brendan]]

Shaver: that dump comes from the GC's mark phase, show it lists only live
objects.  Lotta functions.  More in 0.9.1, good results so far, but it's not
clear what to do other than lazy load their precompiled XDR, but that's not
gonna make 0.9 either.

/be

Comment 12

[Brendan Eich [:brendan]]

Nothing happened in 0.9.1 yet due to this data, and there's only one day left.

/be

Comment 13

[Brendan Eich [:brendan]]

FastLoad work should try to deserialize the infrequently-used function objects
on demand, but not for 0.9.3.

/be

Comment 14

[Brendan Eich [:brendan]]

Oink.

/be

Comment 15

[Brendan Eich [:brendan]]

This bug is too vague (JS has instrumentation, are we using it?  Maybe we just
need shaver's perl-fu).  I'm likely to close it soon, but I'll keep it into
0.9.6 while bug 62164 gets fixed.

/be

Comment 16

[Brendan Eich [:brendan]]

Bumping.

/be

Comment 17

[Brendan Eich [:brendan]]

Hey dbaron, could you r= the forthcoming reduced patch?  I'd like to get it in
for 0.9.7.

/be

Comment 18

[Brendan Eich [:brendan]]

Attachment: updated patch to js/src and dom/src/base

Comment 19

[Brendan Eich [:brendan]]

Ugh, dbaron pointed out that I should impute GC_MARK_DEBUG from NS_TRACE_MALLOC
also in js/src/xpconnect/src/Makefile.in, because nsXPConnect.cpp has two #ifdef
GC_MARK_DEBUG sections.  That suggests that I should move the guts of
dump_xpc.cpp into an existing xpconnect/src/*cpp file and put the extern "C"
prototype in some new or existing .h file.  Jband, can you advise?

/be

Comment 20

[Brendan Eich [:brendan]]

Well, I suck for not landing an updated version of the patch, but now at least I
can get the dump_xpc.* stuff properly integrated into xpconnect.

/be

Comment 21

[John Bandhauer]

xpcdebug.cpp would be the place for the code. xpconnect has no non-generated
public headers. So, a C++ section of nsIXPConnect.idl seems like the place for
anything that needs to be #includ'd by (say) the DOM. The xpconnect makefiles do
not currently have an automated way to turn GC_MARK_DEBUG on and off. I've done
that by hand in the past. A may to say that more globally at build time from an
env var would be handy.

Comment 22

[John Bandhauer]

FWIW, I'm not real excited about the RTTI aspect of this. It *does* give nice
concise (and more or less precise) info on the class of the actual wrapped
object. But, it requires 'funky' builds. XPCWrappedNative::ToString (or some new
subset) get's you most of the way to knowing about the class of the wrapped
object without RTTI. It will only be substandard on objects that we either have
not QI'd to a specific enough interface or when a fairly generic interface (that
is implemented by numerous concrete classes) is used.

Comment 23

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Maybe you should use the nsGetTypeName in xpcom/base/nsTypeInfo.cpp?

Comment 24

[timeless]

is dumpxpc.h really Copyright (C) 2000 Netscape Communications Corporation. the 
patch i was looking at had no - lines, and work for this bug is all in the year 
2001.

i'm sure i'm asleep, 
http://lxr.mozilla.org/mozilla/search?string=GetXPCObjectClassName has 1 hit 
(caller inside ifdef) and google has no hits.

Comment 25

[Simon Fraser [no longer active]]

> FWIW, I'm not real excited about the RTTI aspect of this

So can we #ifdef it depending on whether the current platform is building with 
RTTI on?

Comment 26

[Brendan Eich [:brendan]]

Shaver, you want this bug?

I foresee merge woes, but we should figure out the right way to get the dump_xpc
stuff in.

/be

Comment 27

[Brendan Eich [:brendan]]

I don't have time for more here, and don't feel a strong need to get this in.  I
expect shaver may do some GC work on a branch soon, to be landed in 1.1, that
will standardize and cleanly integrate the patch here.  Moving out.

/be

Comment 28

[Brendan Eich [:brendan]]

Moving out.

/be

Comment 29

[Brendan Eich [:brendan]]

Moving out, some of these may move to 1.3alpha shortly.

/be

Comment 30

[Brendan Eich [:brendan]]

Fixing TM.

/be

Comment 31

[Brendan Eich [:brendan]]

Disowning.

/be

Comment 32

[Bob Clary [:bc] (inactive)]

-> default qa

Comment 33

[Robert Sayre]

We have lots of instrumentation now, from newer bugs added in the Firefox 3 era used to add the cycle collector.