524540 - Deadlock when Abort induced during GC::Collect

Status: VERIFIED FIXED

(Tamarin Graveyard :: Garbage Collection (mmGC), defect, P2)

Description

[Brent Getlin]

User-Agent:       Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Build Identifier: 

If the memory hardlimit is induced, we attempt to iterate through all active GC instances to invoke a collection.  If GC::Collect is invoked "out of line" from the current thread's GC::stackEnter, and then Aborts due to memory allocation, the m_gcLock on that GC is never released. 

Reproducible: Sometimes

Comment 1

[Brent Getlin]

Attachment: Patch

P4 change:594536

Comment 2

[Lars T Hansen]

Substantive:

- the comment in GCHeap.cpp about "collecting on another thread's gc" is
  misleading, because GCs don't belong to threads, and indeed the only way
  that ef->m_collectingGC can be non-NULL is that that GC is not active on
  any thread in the conventional sense.  What is actually going on is that
  the calling thread is holding a lock for a GC that's not currently
  active on this thread, and that lock must be broken.

Stylistic:

- broken indentation in new code in GCHeap.cpp (one tab too many; tab stops
  are 4 places when used at all)
- the "NULL == whatever" style in the new code in GC.cpp needs to be changed
  (that style is not used in the VM)

Comment 3

[Lars T Hansen]

Comment on attachment 408459 [details] [diff] [review]
Patch

I'll make the adjustments before landing.

Comment 4

[Lars T Hansen]

Attachment: Identical patch, but in proper format

Comment 5

[Lars T Hansen]

Attachment: Slightly reengineered following comments

Comment 6

[Lars T Hansen]

redux changeset:   2885:d45e4c075424

Comment 7

[Chris Peyer]

Engineering work item.  Marking as verified.