There's a OOM in content process bp-b4853361-f3cf-4609-b155-d20b82161021, which failed allocating memory for enlarging the IPC pending message queue. The report has: NumberOfPendingIPC 65535 TopPendingIPCCount 65351 TopPendingIPCName ??? TopPendingIPCType 0x440003 which means in the pending 65535 messages, 65351 of them are PCompositorBridge::Msg_DidComposite__ID. :mattwoodrow has some ideas about what can cause this in bug 1266517 comment 23, but basically we should avoid it from happening.
It seems that PCompositorBridge::Msg_DidComposite__ID is used only for MozAfterPaint and compression (IPDL compress/compressall) can't be applied. I am not sure what else we can do here, maybe simply drop the messages if we're not listening for MozAfterPaint? Though it will still have issues when there's a listener.
:kanru told me graphics is a more appropriate component.
Too late for firefox 52, mass-wontfix.