752086 - Create an Anti-StoreBuffer for storing deleted entries

Status: RESOLVED DUPLICATE of bug 764962

(Core :: JavaScript Engine, defect)

Description

[Terrence Cole [:terrence]]

The problem: whenever we have HeapPtrs (which implicitly store their location into the StoreBuffer when updated) stored in memory managed outside the GC, we end up with the potential of having the wrong pointers stored in the remembered set.  If the wrong pointers are in the remembered set, we will touch memory we don't own during GC when updating pointers in the remembered set.

Concrete Example:
  * We have a Set s filled with HeapPtrObject
  * s decides to grow itself after s.add(HeapPtrObject(nurseryPtr))
  * s reallocs the memory where nurseryPtr is stored, moving the allocation
  * The GC runs, and attempts *(s[nurseryPtr])
  * Kaboom!

The solution we use currently is to not trigger the post barriers implicitly -- i.e. use RelocatablePtrObject instead of HeapPtrObject.  We then have to do manual post-barriers by, for example, storing the map and key separately and re-doing the lookup when needed.  The downsides of this approach are that it requires a fair bit of code, it is spread out throughout the implementation, it requires detailed knowledge of the GC's workings, and forces us to keep around a secondary StoreBuffer that is both less efficient and rarely used.

An easier to use solution would be to have RelocatablePtrObject able to "unremember" itself from the remembered set when it moves: e.g. ~RelocatablePtrObject would delete its address from the store buffer.  The downside to this is that it is extremely slow because of the required linear scan and compaction on every removal.

The solution to this is, of course, the same as the solution to maintaining the remembered set in the first place: a StoreBuffer, or more aptly for this case: an AntiStoreBuffer.  Each DirectBuffer in the StoreBuffer should be combined with a matching DirectBuffer to record removals and appropriate machinery should be added to Relocatable* to add entries to it on destruction.

The complex part is how you combine the two buffers to get the correct remembered set: if memory gets moved, then moved back, it is important not to remove too few or too many elements from the buffer for each entry in the anti-buffer.  I will leave an algorithm implementation for a future version of the writer.

Comment 1

[Terrence Cole [:terrence]]

This is implemented in:
https://hg.mozilla.org/projects/holly/rev/ea3114c6031a
But still needs some elaboration.

Comment 2

[Terrence Cole [:terrence]]

This made it in with bug 764962.