Reduce the number of copies involved in sending IPC Messages

RESOLVED FIXED in Firefox 28

Status

()

defect
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: khuey, Assigned: bent.mozilla)

Tracking

unspecified
mozilla29
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite ?

Firefox Tracking Flags

(blocking-b2g:1.3+, firefox27 wontfix, firefox28 fixed, firefox29 fixed, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 fixed)

Details

(Whiteboard: [caf priority: p3][CR 593782][MemShrink])

Attachments

(3 attachments)

bent noticed that we make a copy of every IPC message before we send it.[0]  That's going to be terrible for the performance and memory usage of anything that does heavy IPC traffic, and could lead to OOM of things like the Twitter app.

[0] http://hg.mozilla.org/mozilla-central/annotate/f99c42bb080e/ipc/glue/MessageChannel.cpp#l248
I wonder if bug 957498 has the same root cause.
I think we should aim on big memory chunks, for example, big strings and memory blocks.  For small data, copying is not really serious.  I had look into the problem of big memory chunks of IPC, and found we could use shared memory to avoid copying.  But, it is not easy to manage life cycle of memory map, and you would mmap the memory correctly at first moment.  But, for most situations, you don' know the buffer would be used for IPC.  For example, sending content of JS strings through IPC.  So, start to write a kernel module to implement shared memory with COW. (see the following link.)  You could assigned a range as shared memory, then the module would return a fd.  Then, the fd could be sent to the peer of an IPC channel.  The peer process could mmap the fd to its memory space, so two processes shared the same list of pages.  But, any changes to the pages would trigger a copying for COW.


This approach requires an additional kernel module, but more easy to use and apply.

https://bitbucket.org/thinker/memcow
I had described something wrong.  The peer process does not mmap the fd, it calls ioctl() to map pages into a block of anonymous pages.  So, the app code don't need to manage memory map.
Avoids the copy for our stack tracking. We currently only need a few things from the message, not the entire payload.
Assignee: nobody → bent.mozilla
Status: NEW → ASSIGNED
Attachment #8357994 - Flags: review?(benjamin)
This is just a mechanical change to move the stack classes to the cpp to cut down on the rebuilds needed for changes to these classes.
Attachment #8357997 - Flags: review?(benjamin)
(In reply to Thinker Li [:sinker] from comment #2)
Yeah, this bug isn't so much about copying between processes, just copying unnecessarily on the stack.
Hi ben,

I am guessing that this bug may provide us solution for https://bugzilla.mozilla.org/show_bug.cgi?id=956105 . I tried to apply both of your patches . But It gives some conflicts . Could you please rebase your patches again :)
Flags: needinfo?(bent.mozilla)
Here's one for gecko 28 that includes both of the trunk patches. I didn't see any conflicts applying the others though...
Flags: needinfo?(bent.mozilla)
Attachment #8357994 - Flags: review?(benjamin) → review+
Attachment #8357997 - Flags: review?(benjamin) → review+
https://hg.mozilla.org/mozilla-central/rev/781816c1eefd
https://hg.mozilla.org/mozilla-central/rev/356ce17c2c45
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla29
Blocks: 942267
blocking-b2g: --- → 1.3?
Whiteboard: [MemShrink] → [R 593782][MemShrink]
Whiteboard: [R 593782][MemShrink] → [CR 593782][MemShrink]
blocking-b2g: 1.3? → 1.3+
Flags: in-testsuite?
Whiteboard: [CR 593782][MemShrink] → [caf priority: p3][CR 593782][MemShrink]
You need to log in before you can comment on or make changes to this bug.