Mechanisms and test cases for MMgc performance regression testing

VERIFIED FIXED in flash10.1

Status

VERIFIED FIXED
10 years ago
9 years ago

People

(Reporter: lhansen, Assigned: lhansen)

Tracking

unspecified
flash10.1
Bug Flags:
in-testsuite +

Details

Attachments

(12 obsolete attachments)

(Assignee)

Description

10 years ago
We should be regression testing MMgc performance and memory consumption.  Doing so involves:

a) A notion of the performance data that might be interesting to examine.  
   This includes at least total time spent in the GC, pause times, and peak
   memory numbers for the GC and the underlying block manager, maybe even
   the history of memory growth for the benchmark.
b) A mechanism for gathering performance data.
c) Processing the performance data to look for regressions and track performance
   over time.
d) A set of credible benchmarks programs.

The selection of benchmarks can be tricky; most programs do not (or should not) stress the GC very much.  Ergo programs that do are rather atypical.  And pure AS3 apps are often not at all representative for programs that run in the Flash Player or in AIR.
(Assignee)

Comment 1

10 years ago
Created attachment 385996 [details] [diff] [review]
Better mechanisms for profiling gross gc behavior
Attachment #385996 - Flags: review?(treilly)
(Assignee)

Comment 2

10 years ago
Created attachment 386013 [details] [diff] [review]
Better code still

Cleans up formatting so that it is easy to parse, separates concerns, adds a little information on eg ZCT reap times.
Attachment #385996 - Attachment is obsolete: true
Attachment #385996 - Flags: review?(treilly)
(Assignee)

Comment 3

10 years ago
Created attachment 386015 [details]
sfib.as

This is a simple (and completely synthetic) program that pushes the zct reaping machinery.  Pretty much all objects are strings, the allocation rate is high, and lifetimes are very short for most objects.  Consequently most of the time is spent just allocating strings and reaping them from the ZCT; the garbage collector runs but rarely.

With current code on the Fuze the running time is approx 88 seconds and about 13 seconds is spent in the ZCT reaper (and object deallocation, they're intertwined).  Pause times are moderate, though, peaking around 150ms on the device.
(Assignee)

Comment 4

10 years ago
Comment on attachment 386013 [details] [diff] [review]
Better code still

The code was pushed (redux changeset 2077:4720585bd557) but a review would still be good.  I suspect there may be some overlap with memstats, which we may wish to identify and clean up by and by.
Attachment #386013 - Flags: review?(treilly)
(Assignee)

Comment 5

10 years ago
Created attachment 386094 [details]
ofib.as

Test case carried forward from bug #486504: creates a huge structure that cannot be reclaimed by reference counting but requires full GC.
(Assignee)

Comment 6

10 years ago
Created attachment 386095 [details]
ofib-rc.as

Test case carried forward from bug #486504.  Creates a huge tree that can be reclaimed by reference counting alone when it becomes garbage; puts pressure on ZCT reaping and may cause significant ZCT reap pauses.
(Assignee)

Updated

10 years ago
Target Milestone: --- → flash10.x
(Assignee)

Comment 7

10 years ago
Comment on attachment 386015 [details]
sfib.as

test/performance/mmgc/sfib.as, redux changeset 2079
Attachment #386015 - Attachment is obsolete: true
(Assignee)

Comment 8

10 years ago
Comment on attachment 386094 [details]
ofib.as

test/performance/mmgc/ofib.as, redux changeset 2079
Attachment #386094 - Attachment is obsolete: true
(Assignee)

Comment 9

10 years ago
Comment on attachment 386095 [details]
ofib-rc.as

test/performance/mmgc/ofib-rc.as, redux changeset 2079
Attachment #386095 - Attachment is obsolete: true
(Assignee)

Comment 10

10 years ago
Created attachment 386491 [details] [diff] [review]
Additional profiling code (requires attachment #386013 [details] [diff] [review])
Attachment #386491 - Flags: review?(treilly)
(Assignee)

Comment 11

10 years ago
(In reply to comment #10)
> Created an attachment (id=386491) [details]
> Additional profiling code (requires attachment #386013 [details] [diff] [review])

Pushed, with minor bug fixes, as redux changeset 2082:5f4f0c5c596b.
(Assignee)

Comment 12

10 years ago
Created attachment 386526 [details] [diff] [review]
Yet more profiling code (requires attachment #386491 [details] [diff] [review])

Refactoring; record peak heap and gc allocation; dump interesting stats at program shutdown.
(Assignee)

Comment 13

10 years ago
Comment on attachment 386526 [details] [diff] [review]
Yet more profiling code (requires attachment #386491 [details] [diff] [review])

was pushed as redux changeset: 2085:32e1031b955c; review would be good anyhow.
Attachment #386526 - Flags: review?(treilly)

Comment 14

10 years ago
Comment on attachment 386526 [details] [diff] [review]
Yet more profiling code (requires attachment #386491 [details] [diff] [review])

I think GetMaxHeapAlloc
Attachment #386526 - Flags: review?(treilly) → review+

Updated

10 years ago
Attachment #386013 - Flags: review?(treilly) → review+

Updated

10 years ago
Attachment #386491 - Flags: review?(treilly) → review+
(Assignee)

Updated

10 years ago
Attachment #386491 - Attachment is obsolete: true
(Assignee)

Updated

10 years ago
Attachment #386013 - Attachment is obsolete: true
(Assignee)

Updated

10 years ago
Attachment #386526 - Attachment is obsolete: true
(Assignee)

Comment 15

10 years ago
(In reply to comment #14)
> (From update of attachment 386526 [details] [diff] [review])
> I think GetMaxHeapAlloc

I think so too.

Proposed API:

  void GetMaxHeapAlloc(uint64_t& heap_peak, uint64_t& private_peak)

where heap_peak is the max amount allocated to the heap (in bytes) and private_peak is the amount of private memory (in bytes) recorded at the time when the heap was at its peak.
(Assignee)

Comment 16

10 years ago
Created attachment 388357 [details] [diff] [review]
Tentative patch for GetMaxHeapAlloc
(Assignee)

Comment 17

10 years ago
Created attachment 388522 [details] [diff] [review]
GetMaxTotalHeapSize
Attachment #388357 - Attachment is obsolete: true
Attachment #388522 - Flags: review?(treilly)

Updated

10 years ago
Attachment #388522 - Flags: review?(treilly) → review+
(Assignee)

Comment 18

10 years ago
Comment on attachment 388522 [details] [diff] [review]
GetMaxTotalHeapSize

redux changeset:   2163:cce1a6d531a9
Attachment #388522 - Attachment is obsolete: true
(Assignee)

Comment 19

10 years ago
Created attachment 389527 [details] [diff] [review]
Workaround for WinMo crasher caused by GCMaxTotalHeapSize

Haven't had time to test this on WinMo, should be safe.
Attachment #389527 - Flags: review?(rreitmai)

Comment 20

10 years ago
Comment on attachment 389527 [details] [diff] [review]
Workaround for WinMo crasher caused by GCMaxTotalHeapSize

fixes winmo issue
Attachment #389527 - Flags: review?(rreitmai) → review+
(Assignee)

Comment 21

10 years ago
Comment on attachment 389527 [details] [diff] [review]
Workaround for WinMo crasher caused by GCMaxTotalHeapSize

redux changeset:   2168:38eea73bdb29
Attachment #389527 - Attachment is obsolete: true
(Assignee)

Comment 22

10 years ago
Created attachment 389676 [details] [diff] [review]
Measure RC and ZCT traffic as well as allocation volumes (initial attempt)

Based on redux 2168:38eea73bdb29.
(Assignee)

Comment 23

10 years ago
Created attachment 389740 [details] [diff] [review]
Measure RC and ZCT traffic as well as allocation volumes

More logging code.  Refcount monitoring turned out to be expensive enough that it gets its own ifdef.  I've also profiled the cost of policy profiling in general; looks like +5% execution time in the avmshell on MacOS X for gcbench.  So it's worth our bother not to enable it by default in production runs of the Player.
Attachment #389676 - Attachment is obsolete: true
Attachment #389740 - Flags: review?(treilly)

Updated

10 years ago
Attachment #389740 - Flags: review?(treilly) → review+
(Assignee)

Comment 24

10 years ago
Comment on attachment 389740 [details] [diff] [review]
Measure RC and ZCT traffic as well as allocation volumes

redux changeset:   2184:d2c55524e3ca
Attachment #389740 - Attachment is obsolete: true
(Assignee)

Comment 25

10 years ago
Probably done for now.
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED

Updated

10 years ago
Flags: in-testsuite?

Updated

9 years ago
Status: RESOLVED → VERIFIED

Comment 26

9 years ago
Performance tests were pushed to redux changeset:   2079:572e7cb6492d

Tracking bug for enabling the tests in buildbot: Bug 559439
Flags: in-testsuite? → in-testsuite+
You need to log in before you can comment on or make changes to this bug.