412309 - GCHeap fragmentation issues

Status: VERIFIED FIXED

(Tamarin Graveyard :: Garbage Collection (mmGC), defect)

Description

[Tommy Reilly]

MMgc's GCHeap has a couple problems surfaced by Apollo's higher memory demands, long running high-definition H264 video applications and large Flex applications that load/unload parts of themselves repeatedly on demand:

1) Decommit is only functional on Windows.  We switched from munmap to mprotect(PROT_NONE) to fix an OSX leak (calling mmap on an existing region caused a virtual memory leak and eventual crash).   This fixed the leak but broke decommit, ie mprotect doesn't make the memory available.  Decommiting but keeping reserved a region of memory doesn't appear to be supported (can vm_remap do it?).  We have to munmap it, mmap it again and deal with the possible failure of someone stealing the memory from under us.

2) GCHeap doesn't sort freelists and external fragmentation can result.  To address this we should do two things:

  a) direct "huge" allocations (> 128/256 kb) right to the OS
  b) sort freelists by region age, ie force the use of older regions (even if a new region has a better fit), the idea is to allow higher/newer memory addresses to become free.
  c) for b to matter we need to have the concept of HeapBlock shrinkage, ie if we aren't gonna completely free memory (which we never do right now) it doesn't matter where we put blocks in the reserved memory space.  Sorting and favoring lower addresses is meant to free up space at the end of our VM region so we can shrink it.  

So this is a 3 phase fix:

1) replace DecommitMemory on mac with munmap/mmap and deal with failure (treat the blocks as in use?)
2) redirect huge allocations to the OS
3) cook up a ShrinkHeap analog to ExpandHeap that free's part of the HeapBlock array and gives back virtual memory

I'm not sure about #3.   I think a better solution maybe to have each GC get memory in megachunks (1mb?) and completely own these chunks.  jemalloc calls these arenas and stores them in thread locals for lock free thread local allocation.   Then GCHeap becomes a megablock manager and anytime a megablock becomes completely free it gets returned to the OS.  Need to think about it more.  It may be that we want a different solution for the standalone/Apollo case where all memory requests go through GCHeap and the plugin case where GCHeap is just one of many players and player instances come and go frequently.

Comment 1

[Tommy Reilly]

instead of mprotect in Decommit on OS X we should use vm_* API's there we can vm_deallocate and vm_map the decommit region and flag the pages as dead (in use?) if vm_map fails (because we context switched and someone else got the memory).

Comment 2

[Tommy Reilly]

What this means is that we can avoid the ShrinkHeap notion and any accompanying freelist sorting complexity in order to give memory back to the OS.  We'd still need to go there to shrink virtual memory but I'm not convinced the ability to shrink virtual memory is important, especially in Apollo/standalone where the GCHeap is the source of all memory for the process.

Comment 3

[Tommy Reilly]

Hmm, maybe madvise with MADV_FREE is really what we want...

Comment 4

[Tommy Reilly]

These issues are fixed in tamarin-redux

Comment 5

[Chris Peyer]

removing QRB request, bug verified