1898647 - Seperate the cache of unallocated memory into: dirty, clean, unmapped.

Status: NEW

(Core :: Memory Allocator, enhancement)

Description

[Paul Bone [:pbone]]

jemalloc uses a single red-black tree to look for unallocated pages, they're sorted by size so it can to "best fit" allocation. But not by "status". So if in the tree there is a unmapped block of 4 pages before a dirty block of 4 pages. and the allocator is looking for at least two pages, it may map in the new memory (increasing RSS) rather than use 2 dirty pages (decreasing overheads). This is especially true if for example there was a block of 6 dirty pages which will always be sorted after 4 unmapped pages.

We should modify jemalloc to create multiple free lists for: dirty, clean, unmapped and check them in sequence. This could improve Firefox's memory footprint and performance.

Comment 1

[Mike Hommey [:glandium]]

Wouldn't changing ArenaAvailTreeTrait to account for the status have roughly the same effect?

Comment 2

[Paul Bone [:pbone]]

(In reply to Mike Hommey [:glandium] from comment #1)

Wouldn't changing ArenaAvailTreeTrait to account for the status have roughly the same effect?

it depends. it's work for the cases where there's a dirty block of 4 pages and an unmapped block of 4 pages.

When there's an unmapped block of 4 pages and a dirty block of 5 pages though I think separate lists would be good. In that case changing the tree trait won't work and I think we'd be happy with that fragmentation rather than mapping in new memory when dirty memory is already there.

What I don't know is if there's a block of 128 pages of dirty memory and we only want to allocate 1 page. Is fragmenting that block okay (we already do that if there's no smaller blocks) Especially since a future allocations are more likely to request a more even 128 pages rather than 127.

Comment 3

[Mike Hommey [:glandium]]

Well, that kind of hits the crux of the issue, which is, is it really more beneficial to use that dirty block of 5 pages or that unmapped block of 4 pages, and how often are you likely to have to make this choice? And as you point out, even if it is beneficial, is there a limit to the size of that dirty block over which you get diminished returns.

Comment 4

[Paul Bone [:pbone]]

Agreed. I think the first step here would be to do some kind of logging/test to see how often different things happen. then maybe to estimate or test for the costs.