1263027 - Large PStorage IPC messages

Status: RESOLVED WONTFIX

(Core :: DOM: Device Interfaces, defect)

Description

[Andrew McCreight [:mccr8]]

The IPC_MESSAGE_SIZE telemetry build landed recently records the sizes of each IPC message. One of the more common IPDL actors for messages of size 844KB or larger is PStorage. For PStorage::Msg_AsyncUpdateItem I see about 20k of these large messages in the last few days, about 14k for PStorage::Msg_LoadItem and about 4k PStorage::Msg_AsyncAddItem. I see more than a thousand of these messages that are 3MB or larger.

IPC messages require contiguous address space, so large messages like this can contribute to OOM crashes, which are currently a problem on the e10s beta experiment. Ideally the IPC layer would be smarter about allocating these, but in the interim it would be nice if clients like PStorage could somehow send smaller messages.

Comment 1

[Honza Bambas (:mayhemer)]

You mean like sending in chunks?

Comment 2

[Andrew McCreight [:mccr8]]

In bug 1263028, Honza said this would be harder. As these large messages are also less common than the http ones, let's just mark this WONTFIX for now. It would probably make more sense to spend that effort on making the IPC layer create segmented messages.

Comment 3

[Honza Bambas (:mayhemer)]

Keys and values are being exchanged in both ways in PStorage.  Loaded to the child as part of preload and then added/updated as content scripts changes them individually back to the parent to be persisted.

In both cases we simply send a whole string as an argument.  This is not that simple to do in chunks in any of the directions.  It's doable, with some larger effort tho.

I agree that that amount of effort should rather be put to doing this in IPC directly.

Comment 4

[Ben Kelly [:bkelly, not reviewing]]

FWIW, the patches in bug 1093357 allow automatic chunking/streaming of an nsIInputStream from child-to-parent.  I have r+ and just need to address some review items and rebase. I hope to land in the next week or two.  Not sure if that would help.