450778 - Need improved memory stats

Status: VERIFIED FIXED

(Tamarin Graveyard :: Garbage Collection (mmGC), defect)

Description

[Tommy Reilly]

We want to optimize the VM's memory usage and ensure going forward we don't introduce changes that increase memory usage (at least without knowing about it).  stats:

private bytes (entire process wired/resident non-shared memory)
gcheap stats (total/used/free)
jit bytes
managed vs. unmanaged bytes
internal fragmentation #s

Anything else?

this will all be exposed via the -Dmemstats option in the shell

Comment 1

[Tommy Reilly]

Attachment: first pass, implements everything for mac

Windows and Linux private bytes calculation to come.  Also fragmentation #'s aren't done, need to account for memory wasted inside each allocation (ie where the size asked for is less than the size class).

Comment 2

[Tommy Reilly]

Sample output from patch:

[mem] ------- gross stats -----
[mem] private 375 (1.5M) 100%
[mem]	 mmgc 288 (1.1M) 76%
[mem]		 unmanaged 12 (48K) 3%
[mem]		 managed 125 (500K) 33%
[mem]		 free 149 (596K) 39%
[mem]	 jit 12 (48K) 3%
[mem]	 other 75 (300K) 20%
[mem] bytes (interal fragmentation) 94 (378K) 69%
[mem] 	managed bytes (internal fragmentation) 88 (352K) 70%
[mem] 	unmanaged bytes (interal fragmentation) 6 (26K) 54%
[mem] -------- gross stats end -----

Comment 3

[Steven Johnson]

Will this work in all builds (Release included) or only Debug/Debugger builds? (If the former, what does the "D" in the flag imply?)

Comment 4

[Tommy Reilly]

The D is just following the convention for shell flags, I always thought it to mean "define" or enable, just occurred to me that it might mean "debug".  This feature works in all builds.

Comment 5

[Tommy Reilly]

Attachment: 2nd pass, diff against latest TC and includes new API's and memlimit feature

Not asking for review yet, will put in windows and linux impl of GetPrivateMemory first

Comment 6

[Tommy Reilly]

Attachment: Windows GetPrivateBytes implemented

Comment 7

[Tommy Reilly]

Attachment: generic non-functional unix impl included

Windows and Mac are implemented, I can't find an api to this on linux, don't want to hold it up any more.  Linux will just report 0 for private bytes until I figure it out.

Comment 8

[Tommy Reilly]

recording some enhancement ideas:

1) show # of objects sweep in sweep messages
2) show # of total objects and calculate average object size for managed and unmanaged memory.

Comment 9

[Dan Schaffer]

triaging linux is still open issue I believe.  works on mac/win.

Comment 10

[Tommy Reilly]

this has been done for while

Comment 11

[Chris Peyer]

removing QRB request, bug verified