683762 - Use ashmem for HTTP memory cache

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect)

Description

[Jason Duell]

Using ashmem to allocate the memory for the memory cache might give us the ability to drop the memory if Android experiences memory pressure, potentially allowing fennec to stay alive longer.

see 

   http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/44a0efec8a336867#

Especially shaver's comment:

--
Fennec on Android could also use ashmem for some things (I've
suggested the image and network caches in the past, maybe all the
canvas/surface memory if we can do that), which would let the OS
discard it for us under memory pressure.

http://elinux.org/Android_Kernel_Features#ashmem

AFAICT the basic model is that before you use something you pin it
based on the handle.  If the pin fails, then you know that the kernel
dropped it, so you reallocate and regenerate.  Can mostly just be
handled like a cache miss, I suspect, and similar to what we would do
on desktop around the regeneration.

Comment 1

[Jason Duell]

See also bug 622723 and bug 595357.  Bug 681525 also probably relevant.

Bug 622723 comment 2 makes it sounds like we don't know yet how to actually have Android dump the ashmem regions when we're under memory pressure.

Note that if we get this working we could also potentiall have a larger RAM cache.  We might even be able to read it directly from the content process (it's shared memory), though that would involve some synchronization work that might be more effort than it's worth.

Comment 2

[(dormant account)]

(In reply to Jason Duell (:jduell) from comment #1)
> See also bug 622723 and bug 595357.  Bug 681525 also probably relevant.
> 
> Bug 622723 comment 2 makes it sounds like we don't know yet how to actually
> have Android dump the ashmem regions when we're under memory pressure.
> 
> Note that if we get this working we could also potentiall have a larger RAM
> cache.  We might even be able to read it directly from the content process
> (it's shared memory), though that would involve some synchronization work
> that might be more effort than it's worth.

I agree on accessing memory directly from child process, just make sure the only access a child does is via memcpy. Sharing memory too much would sacrifice the main benefit of using ashmem(ie that you can drop it at any point). See https://bugzilla.mozilla.org/show_bug.cgi?id=547842#c0 and the paper linked from there.

Comment 3

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

The OS will purge all the ashmem caches when it gets low on memory -- not sure of the exact trigger, I asked the ashmem author.  You can force it for testing with an ioctl(PURGE_ALL_CACHES).

Shared ashmem can still be dropped, I'm *think*, since it can be unpinned by both sides.  Given that ashmem pinning is page-sized, you should still be able to drop a lot even if there is a bunch of activity.  I'll peek at the source and see what I can see.

Comment 4

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Yeah, if you can get the FD for the ashmem mapping on both sides, you can pin and unpin regions, and anything not pinned by either is eligible for destruction at page granularity.

ref: http://android.git.kernel.org/?p=kernel/common.git;a=blob;f=mm/ashmem.c

I was wrong above in comment 3, BTW: the kernel can purge only some of the ashmem mappings in a given pass, though I'm not sure from the slab-shrinking strategy if it actually ever does such a partial purge.

Comment 5

[Jason Duell]

Some more ashmem discussion here:

http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/99f54309b3e83528#

Comment 6

[Jason Duell]

bug 622723 and bug 675259 are relevant here too, though I don't get clear answers to this question from them.

Comment 7

[Jason Duell]

And bug 413947 is basically an alternative solution to this (drop RAM cache every time we're backgrounded).