790762 - Don't use shared memory for cross-thread communication

Status: RESOLVED FIXED

(Core :: IPC, defect)

Description

[Nicolas Silva [:nical]]

On all platforms except B2G we don't actually use cross-process protocols for most things (layer transactions, async-video, etc.). Most likely it is going to take a while before we can use a cross-process architecture like B2G on other platforms because of addons. So in the mean time we should not pay the cost of shared memory When we don't need it.

I have not looked at it yet, but I assume ipdl protocols can know whether they are cross-process or cross-thread protocols. If they are cross process they should still use shared memory (out of process plugins, for example), but all the cross-thread protocols should not use shared memory and allocate using our usual allocator instead.

Comment 1

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

For current shmem users, large buffers, there's basically no difference between shmem and jemalloc, because jemalloc will just mmap() large allocations.  So this overhead you refer to doesn't exist.

For small allocations, there is an overhead.

The change you propose would break IPDL semantics, which allow shmem buffers to be adopted among different protocols.

I suggest that instead, when we have important small-alloc users (none so far), we make a shmem allocator that uses the jemalloc algorithms, instead of trying to special-case in/out-of-process.  A dedicated shmem allocator would outperform jemalloc for cross-thread allocs in every way.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

(In reply to Chris Jones [:cjones] [:warhammer] from comment #1)
> For current shmem users, large buffers, there's basically no difference
> between shmem and jemalloc, because jemalloc will just mmap() large
> allocations.  So this overhead you refer to doesn't exist.

There's the file handling overhead that doesn't exist in jemalloc because the mmaps are anonymous. We also run into file descriptor limits that we wouldn't with jemalloc. I also expect that on Windows the overhead of shared memory (CreateFileMapping etc.) is even higher than VirtualAlloc.

> 
> For small allocations, there is an overhead.
> 
> The change you propose would break IPDL semantics, which allow shmem buffers
> to be adopted among different protocols.

Perhaps it makes sense to have a weaker shmem buffer that doesn't have these semantics. That way we don't have to pay these costs for the most shmem buffers which will not be adopted among different protocols.

Comment 3

[Chris Jones [:cjones] inactive; ni?/f?/r? if you need me]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #2)
> (In reply to Chris Jones [:cjones] [:warhammer] from comment #1)
> > For current shmem users, large buffers, there's basically no difference
> > between shmem and jemalloc, because jemalloc will just mmap() large
> > allocations.  So this overhead you refer to doesn't exist.
> 
> There's the file handling overhead that doesn't exist in jemalloc because
> the mmaps are anonymous. We also run into file descriptor limits that we
> wouldn't with jemalloc. I also expect that on Windows the overhead of shared
> memory (CreateFileMapping etc.) is even higher than VirtualAlloc.
> 

File handling is a small overhead when creating/destroying the mappings, and 0 overhead beyond that.

> > 
> > For small allocations, there is an overhead.
> > 
> > The change you propose would break IPDL semantics, which allow shmem buffers
> > to be adopted among different protocols.
> 
> Perhaps it makes sense to have a weaker shmem buffer that doesn't have these
> semantics. That way we don't have to pay these costs for the most shmem
> buffers which will not be adopted among different protocols.

We already have this, through the magic shared pointer hack that fennec uses.

Comment 4

[Nicolas Silva [:nical]]

wrong bug, sorry