501318 - Mechanisms and test cases for MMgc performance regression testing

Status: VERIFIED FIXED

(Tamarin Graveyard :: Garbage Collection (mmGC), defect)

Description

[Lars T Hansen]

We should be regression testing MMgc performance and memory consumption.  Doing so involves:

a) A notion of the performance data that might be interesting to examine.  
   This includes at least total time spent in the GC, pause times, and peak
   memory numbers for the GC and the underlying block manager, maybe even
   the history of memory growth for the benchmark.
b) A mechanism for gathering performance data.
c) Processing the performance data to look for regressions and track performance
   over time.
d) A set of credible benchmarks programs.

The selection of benchmarks can be tricky; most programs do not (or should not) stress the GC very much.  Ergo programs that do are rather atypical.  And pure AS3 apps are often not at all representative for programs that run in the Flash Player or in AIR.

Comment 1

[Lars T Hansen]

Attachment: Better mechanisms for profiling gross gc behavior

Comment 2

[Lars T Hansen]

Attachment: Better code still

Cleans up formatting so that it is easy to parse, separates concerns, adds a little information on eg ZCT reap times.

Comment 3

[Lars T Hansen]

Attachment: sfib.as

This is a simple (and completely synthetic) program that pushes the zct reaping machinery.  Pretty much all objects are strings, the allocation rate is high, and lifetimes are very short for most objects.  Consequently most of the time is spent just allocating strings and reaping them from the ZCT; the garbage collector runs but rarely.

With current code on the Fuze the running time is approx 88 seconds and about 13 seconds is spent in the ZCT reaper (and object deallocation, they're intertwined).  Pause times are moderate, though, peaking around 150ms on the device.

Comment 4

[Lars T Hansen]

Comment on attachment 386013 [details] [diff] [review]
Better code still

The code was pushed (redux changeset 2077:4720585bd557) but a review would still be good.  I suspect there may be some overlap with memstats, which we may wish to identify and clean up by and by.

Comment 5

[Lars T Hansen]

Attachment: ofib.as

Test case carried forward from bug #486504: creates a huge structure that cannot be reclaimed by reference counting but requires full GC.

Comment 6

[Lars T Hansen]

Attachment: ofib-rc.as

Test case carried forward from bug #486504.  Creates a huge tree that can be reclaimed by reference counting alone when it becomes garbage; puts pressure on ZCT reaping and may cause significant ZCT reap pauses.

Comment 7

[Lars T Hansen]

Comment on attachment 386015 [details]
sfib.as

test/performance/mmgc/sfib.as, redux changeset 2079

Comment 8

[Lars T Hansen]

Comment on attachment 386094 [details]
ofib.as

test/performance/mmgc/ofib.as, redux changeset 2079

Comment 9

[Lars T Hansen]

Comment on attachment 386095 [details]
ofib-rc.as

test/performance/mmgc/ofib-rc.as, redux changeset 2079

Comment 10

[Lars T Hansen]

Attachment: Additional profiling code (requires attachment #386013)

Comment 11

[Lars T Hansen]

(In reply to comment #10)
> Created an attachment (id=386491) [details]
> Additional profiling code (requires attachment #386013 [details] [diff] [review])

Pushed, with minor bug fixes, as redux changeset 2082:5f4f0c5c596b.

Comment 12

[Lars T Hansen]

Attachment: Yet more profiling code (requires attachment #386491)

Refactoring; record peak heap and gc allocation; dump interesting stats at program shutdown.

Comment 13

[Lars T Hansen]

Comment on attachment 386526 [details] [diff] [review]
Yet more profiling code (requires attachment #386491 [details] [diff] [review])

was pushed as redux changeset: 2085:32e1031b955c; review would be good anyhow.

Comment 14

[Tommy Reilly]

Comment on attachment 386526 [details] [diff] [review]
Yet more profiling code (requires attachment #386491 [details] [diff] [review])

I think GetMaxHeapAlloc

Comment 15

[Lars T Hansen]

(In reply to comment #14)
> (From update of attachment 386526 [details] [diff] [review])
> I think GetMaxHeapAlloc

I think so too.

Proposed API:

  void GetMaxHeapAlloc(uint64_t& heap_peak, uint64_t& private_peak)

where heap_peak is the max amount allocated to the heap (in bytes) and private_peak is the amount of private memory (in bytes) recorded at the time when the heap was at its peak.

Comment 16

[Lars T Hansen]

Attachment: Tentative patch for GetMaxHeapAlloc

Comment 17

[Lars T Hansen]

Attachment: GetMaxTotalHeapSize

Comment 18

[Lars T Hansen]

Comment on attachment 388522 [details] [diff] [review]
GetMaxTotalHeapSize

redux changeset:   2163:cce1a6d531a9

Comment 19

[Lars T Hansen]

Attachment: Workaround for WinMo crasher caused by GCMaxTotalHeapSize

Haven't had time to test this on WinMo, should be safe.

Comment 20

[Rick Reitmaier]

Comment on attachment 389527 [details] [diff] [review]
Workaround for WinMo crasher caused by GCMaxTotalHeapSize

fixes winmo issue

Comment 21

[Lars T Hansen]

Comment on attachment 389527 [details] [diff] [review]
Workaround for WinMo crasher caused by GCMaxTotalHeapSize

redux changeset:   2168:38eea73bdb29

Comment 22

[Lars T Hansen]

Attachment: Measure RC and ZCT traffic as well as allocation volumes (initial attempt)

Based on redux 2168:38eea73bdb29.

Comment 23

[Lars T Hansen]

Attachment: Measure RC and ZCT traffic as well as allocation volumes

More logging code.  Refcount monitoring turned out to be expensive enough that it gets its own ifdef.  I've also profiled the cost of policy profiling in general; looks like +5% execution time in the avmshell on MacOS X for gcbench.  So it's worth our bother not to enable it by default in production runs of the Player.

Comment 24

[Lars T Hansen]

Comment on attachment 389740 [details] [diff] [review]
Measure RC and ZCT traffic as well as allocation volumes

redux changeset:   2184:d2c55524e3ca

Comment 25

[Lars T Hansen]

Probably done for now.

Comment 26

[Chris Peyer]

Performance tests were pushed to redux changeset:   2079:572e7cb6492d

Tracking bug for enabling the tests in buildbot: Bug 559439