1280436 - Break reference cycle that results in memory leak

Status: RESOLVED DUPLICATE of bug 1280762

(Core :: Graphics: Layers, defect)

Description

[Kartikaya Gupta (email:[e-mail])]

From bug 1261166 comment 22:

I discovered that there's a cyclical reference which is probably causing the leak.

LayerTransactionParent holds a RefPtr to LayerManagerComposite
LayerManagerComposite holds a RefPtr to CompositorOGL
Compositor holds a bunch of ShmemTextureReadLock instances in mUnlockAfterComposition
The ShmemTextureReadLock instances hold RefPtrs to LayerTransactionParent

(There's a bit more context in that bug).

Comment 1

[Kartikaya Gupta (email:[e-mail])]

Attachment: WIP

Comment 2

[Kartikaya Gupta (email:[e-mail])]

So bug 1176011 didn't fix the crash that's introduced by this patch. Sotaro, do you have any other ideas on what could be going on here? The exact steps to reproduce the crash:

1) Start with current mozilla-central (on OS X)
2) Apply attachment 8763604 [details] [diff] [review] (from bug 1261166)
3) Apply attachment 8763075 [details] [diff] [review] (from this bug)
4) Do a debug build
5) Run ./mach mochitest -f plain --keep-open=false --setpref gfx.use-iosurface-textures=true gfx/layers/apz/test/mochitest/

The test runs but then it crashes on shutdown. If you skip step 3, you will get leaks on shutdown instead.

Comment 3

[Sotaro Ikeda [:sotaro]]

I am going to check it.

Comment 4

[Sotaro Ikeda [:sotaro]]

I am still waiting the debug build complete, but I seems to understand a cause of the crash. A peer of CompositorBridgeChild could be CrossProcessCompositorBridgeParent. Compositor::Destroy() is related to only to CompositorBridgeParent. When CrossProcessCompositorBridgeParent is destroyed, related Shmem also have to be destroyed and untouched after the destroy.

Comment 5

[Sotaro Ikeda [:sotaro]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #2)
> So bug 1176011 didn't fix the crash that's introduced by this patch. Sotaro,
> do you have any other ideas on what could be going on here? The exact steps
> to reproduce the crash:
> 
> 1) Start with current mozilla-central (on OS X)
> 2) Apply attachment 8763604 [details] [diff] [review] (from bug 1261166)
> 3) Apply attachment 8763075 [details] [diff] [review] (from this bug)
> 4) Do a debug build
> 5) Run ./mach mochitest -f plain --keep-open=false --setpref
> gfx.use-iosurface-textures=true gfx/layers/apz/test/mochitest/
> 
> The test runs but then it crashes on shutdown. If you skip step 3, you will
> get leaks on shutdown instead.

I tried several times, but I failed to reproduce the crash. :kats, can you try a patch in Bug 1280762?

Comment 6

[Kartikaya Gupta (email:[e-mail])]

(In reply to Sotaro Ikeda [:sotaro] from comment #5)
> I tried several times, but I failed to reproduce the crash.

Note that it only happens on OS X.

> :kats, can you
> try a patch in Bug 1280762?

That patch seems to fix the issue! I used that patch instead of the one on this bug, since it looks like that already incorporates the changes from this bug.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

This seems like it's effectively a dupe of bug 1280762, so I'll close it as such.

Comment 8

[Sotaro Ikeda [:sotaro]]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #6)
> (In reply to Sotaro Ikeda [:sotaro] from comment #5)
> > I tried several times, but I failed to reproduce the crash.
> 
> Note that it only happens on OS X.

Yes, I checked on OS X.