Open Bug 1979830 Opened 1 year ago Updated 4 months ago

mozjemalloc should not move virtual address space between arenas

Categories

(Core :: Memory Allocator, enhancement)

enhancement

Tracking

()

ASSIGNED

People

(Reporter: pbone, Assigned: jstutte)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

(Keywords: sec-want)

Attachments

(7 external files, 5 obsolete files)

Fixing Bug 1948354 will reduce the chance that an address can be reused during an attack, but it can't guarantee it. If a mozjemalloc chunk (1MB block from which smaller allocations are partitioned) becomes free it may be moved between arenas. I propose allowing arenas to be marked as "secure" during creation, which will prevent mozjemalloc from re-using address space between them and any other arena.

There's a couple of steps:

  • The chunk cache must never release any chunks.
  • Add a secure parameter during arena creation.
  • Secure arenas do not use the chunk cache.
  • Secure arenas must implement their own local cache,
  • if a secure arena is destroyed it's chunk cache must become "orphaned" and no chunk may use it (it may be merged with other deleted chunk caches).
  • The global chunk cache must track huge allocations also.
  • huge allocations in secure arenas must be tracked in that arena's chunk cache.

"chunk cache" may come to mean "extent cache" depending on implementation details.

Group: core-security → dom-core-security
Keywords: sec-want

Is this a (more detailed and more descriptive) dupe of Bug 1446046? If so, we could dupe that one here.

(In reply to Tom Ritter [:tjr] from comment #1)

Is this a (more detailed and more descriptive) dupe of Bug 1446046? If so, we could dupe that one here.

yes. I'll do that now.

Tom, out of caution I made this bug security sensitive, does it actually need to be though?

Thanks.

Flags: needinfo?(tom)
Duplicate of this bug: 1446046
Group: dom-core-security
Flags: needinfo?(tom)
See Also: → 2029736

SplitAndAllocRun moves a chunk to the back of mChunksDirty whenever
dirty pages are consumed and the chunk stays dirty, so chunks that
have stopped seeing reuse drift to the front. Purge drops its explicit
"prioritise the spare's dirty pages" branch — the spare has no
mRunsAvail entry, never gets warmed, and naturally ends up at the
front.

Pure data-structure refactor. Replace the single mSpare slot with a
DoublyLinkedList<arena_chunk_t, SpareChunkListTrait> mSpareChunks
plus an mNumSpareChunks counter. Behavior is identical: the list
holds at most one element (DemoteChunkToSpare evicts the oldest when
size > 1), so reuse and dealloc patterns match the prior single-spare
semantics.

The list shape sets up the per-arena retention pool used by Part 3.

Drive-by: rename arena_chunk_t::mChunksDirtyElim to mChunksDirtyElement
and fix the typo mChunksMavisedElim → mChunksMadvisedElement, and add
the const Get() overload to DirtyChunkListTrait for symmetry with the
new SpareChunkListTrait.

Allow each arena to retain a configurable number of empty chunks
between alloc/free cycles, so workloads with chunk-level churn don't
pay mmap/unmap round trips when the chunk count oscillates near the
working-set boundary.

arena_params_t::mMinSpareRetention sets the floor; default 1.
UpdateMaxDirty drops the effective floor to 0 on background.

Purge gains a preamble that drains excess spares oldest-first:
PurgeUnconditional drains to 0, otherwise to mMinSpareRetention. On
64-bit the drain also gates on ShouldContinuePurge.

DemoteChunkToSpare skips its eviction path when lazy purge is enabled
— Purge's preamble handles the drain instead.

PurgeStats gains spares_drained / spares_retained, surfaced via
PurgeArenaMarker.

No arena opts in to a non-default floor in this patch; Part 4 opts in
the default arena and the JS arenas.

The default arena and the four JS arenas (MallocArena,
BackgroundMallocArena, ArrayBufferContentsArena, StringBufferArena)
set mMinSpareRetention = 4 on 64-bit. 32-bit stays at the default of 1
(address-space budget).

These arenas see the most chunk-level alloc/free churn, so a small
warm pool of empty chunks meaningfully reduces mmap/unmap rate.

Move chunk_record / chunk_recycle (and the gChunksBySize /
gChunksByAddress trees they walk) into a reusable ExtentPool class.
Drop the global instance — no caller instantiates an ExtentPool yet.
Part 6 wires it in as the cold-tier huge-allocation retention for
isolated arenas, where coalesce + best-fit split is needed because
huge allocations are variable-sized.

Cross-arena VA reuse via the global cache is what bug 1979830 wants
to retire for security; with Parts 2-3's per-arena retention covering
the short-lived reuse window, the cache provided little measurable
benefit anyway (entries were purged on insertion, so reuse paid
recommit/fault).

ExtentPool is deliberately the cold tier: Record decommits on entry,
Recycle commits on exit. The hot tier (mSpareChunks) keeps headers and
dirty pages around for cheap reuse without commit or fault.

BaseAlloc::release_whole_chunk now calls pages_unmap directly. It only
ever deposited into the cache, never withdrew, so the change is a
straight replacement of the post-recycle-limit fallback path.

Adds arena_params_t::mIsolated (default false). Isolated arenas:

  • Never drain mSpareChunks (DemoteChunkToSpare and Purge's preamble
    short-circuit; UpdateMaxDirty doesn't drop the retention floor on
    background).
  • Route huge_dalloc through a per-arena ExtentPool
    (arena_t::mIsolatedHugeExtents). huge_palloc consults it before
    arena_chunk_alloc; gChunkRTree is updated accordingly.
  • Cannot be disposed via moz_dispose_arena — MOZ_RELEASE_ASSERT.
    Disposable isolated arenas need a process-wide graveyard pool
    (pbone's design in bug 1979830 comment 0) to park retained VA
    without leaking arena_t bookkeeping per dispose. Until that
    lands, the combination is unsupported.

The address-space cost of isolation is surfaced in jemalloc_stats as
mapped_isolated (sum of mapped over isolated arenas), and exposed
to about:memory as isolated-arenas/mapped in the "Other Measurements"
section (the same bytes are already counted under heap, so the metric
sits outside the heap tree to avoid double-summing).

Part 7 opts in the JS arenas; no arena in this patch sets the flag.
js::ShutDownMallocAllocator currently doesn't invoke moz_dispose_arena
on the JS arenas (the calls are commented out), so the disposal assert
isn't reachable from the JS opt-in this stack adds.

Set mIsolated = true on MallocArena, BackgroundMallocArena,
ArrayBufferContentsArena and StringBufferArena on 64-bit.

Drops their mMinSpareRetention = 4 from Part 1 — isolated arenas never
drain mSpareChunks, so the explicit retention floor is redundant.

32-bit stays opted out (address-space budget).

SplitAndAllocRun moves a chunk to the back of mChunksDirty whenever
dirty pages are consumed and the chunk stays dirty, so chunks that
have stopped seeing reuse drift to the front. DemoteChunkToSpare
pushFront's the new spare so it sits at the coldest end with no
allocations to warm it. Purge drops its explicit "prioritise the
spare's dirty pages" branch and just popFront's — same observable
purge order, with non-spare chunks now ordered by LRU instead of
FIFO.

Attachment #9574700 - Attachment is obsolete: true
Assignee: nobody → jstutte
Attachment #9574692 - Attachment description: WIP: Bug 1979830 - Part 1: Use LRU ordering for mChunksDirty.popFront() in Purge r?pbone → Bug 1979830 - Part 1: Use LRU ordering for mChunksDirty.popFront() in Purge r?pbone
Status: NEW → ASSIGNED
Attachment #9574693 - Attachment description: WIP: Bug 1979830 - Part 2: Replace arena_t::mSpare with mSpareChunks list r?pbone → Bug 1979830 - Part 2: Replace arena_t::mSpare with mSpareChunks list r?pbone
Attachment #9574694 - Attachment description: WIP: Bug 1979830 - Part 3: Configurable spare-chunk retention floor and Purge drain preamble r?pbone → Bug 1979830 - Part 3: Configurable spare-chunk retention floor and Purge drain preamble r?pbone
Attachment #9574695 - Attachment description: WIP: Bug 1979830 - Part 4: Opt high-churn arenas into 4 retained spares r?pbone → Bug 1979830 - Part 4: Opt high-churn arenas into 4 retained spares r?pbone
Attachment #9574696 - Attachment description: WIP: Bug 1979830 - Part 5: Refactor chunk recycling into ExtentPool, drop the global instance r?pbone → Bug 1979830 - Part 5: Refactor chunk recycling into ExtentPool, drop the global instance r?pbone
Attachment #9574697 - Attachment description: WIP: Bug 1979830 - Part 6: Add isolated-arena mechanism (no callers yet) r?pbone → Bug 1979830 - Part 6: Add isolated-arena mechanism (no callers yet) r?pbone
Attachment #9574698 - Attachment description: WIP: Bug 1979830 - Part 7: Mark JS arenas as isolated r?pbone → Bug 1979830 - Part 7: Mark JS arenas as isolated r?pbone

Wires the isolated-arena graveyard (gIsolatedGraveyard, added in
Part 6) into the idle memory cleanup loop so its VA is eventually
returned to the OS.

ExtentPool gains:

  • mInsertedAt on each extent_node_t, updated on coalesce.
  • mByInsertedAt: a FIFO DoublyLinkedList ordered by insertion time,
    maintained alongside the existing size/address trees.
  • EvictOlderThan(aThreshold, aPeekOnly, aKeepGoing): walks the FIFO
    front and unmaps entries older than the threshold. Returns Done when
    no eligible entries remain (including when the pool is empty) and
    NeedsMore after each unmap, compatible with the existing idle-runner
    slice protocol. WantsLater is never returned: the graveyard
    threshold is O(minutes) while the runner's WantsLater polling is
    O(seconds), so signalling WantsLater would keep the runner alive
    for the full graveyard age unnecessarily.
  • OldestInsertedAt(): for the stats field below.
  • DrainAll(): unmaps all entries unconditionally; called from
    jemalloc_free_excess_dirty_pages on memory pressure.
  • RecordDecommitted(): private helper that skips pages_decommit,
    used by DrainInto to move entries without re-decommitting and to
    preserve their original insertion timestamps.

MayPurgeSteps now has a Phase 2 that calls EvictOlderThan when Phase 1
finds no ready arena purge work (arenaResult is Done or WantsLater).
The new aGraveyardAgeMinutes parameter carries the threshold, read from
the memory.lazypurge.graveyard_age_minutes pref (default 15).

jemalloc_stats_t adds graveyard_isolated_oldest_ms, exposed to
about:memory as heap-isolated-graveyard-oldest-ms, to make the
graveyard age observable alongside its size.

Tests cover age-gated eviction, peek-only mode, memory-pressure drain,
and the oldest-ms stats field.

Per-document DOM arenas are short-lived: each is disposed when its
document is destroyed, at which point mIsolatedExtents drains into the
process-wide graveyard (Part 6) so the VA is quarantined rather than
returned to the OS.

Isolated on 64-bit only, matching the JS arena policy.

When MayPurgeSteps visits an isolated arena for dirty-page work and
PurgeLoop completes (Done or ReachedThreshold), piggyback an
EvictOlderThan call on that same slice to evict any mIsolatedExtents
entries that have aged past the graveyard threshold. This reuses the
threshold already computed for Phase 2 (graveyard eviction) and does not
disturb mOutstandingPurges selection or ordering.

jemalloc_free_excess_dirty_pages gains a DrainAll pass over all live
isolated arenas' mIsolatedExtents, mirroring the graveyard DrainAll.

One new gtest verifies that jemalloc_free_excess_dirty_pages drains a
live isolated arena's extent pool unconditionally.

The default arena is long-lived and sees the highest allocation churn
across a Firefox process lifetime. Isolating it on 64-bit builds means
freed chunks are parked in mIsolatedExtents for fast intra-arena recycling
rather than unmapped to the OS, reducing mmap/munmap syscall pressure.
Age-gated eviction (Part 10) reclaims VA that has not been recycled within
the configured threshold.

Isolated on 64-bit only (VA is scarce on 32-bit), co-located with the
existing mMinSpareRetention guard.

Attachment #9591204 - Attachment description: WIP: Bug 1979830 - Part 8: Age-gated graveyard eviction via MayPurgeSteps r?pbone → Bug 1979830 - Part 8: Age-gated graveyard eviction via MayPurgeSteps r?pbone
Attachment #9591205 - Attachment description: WIP: Bug 1979830 - Part 9: Mark DOM arenas as isolated r?pbone → Bug 1979830 - Part 9: Mark DOM arenas as isolated r?pbone
Attachment #9591206 - Attachment description: WIP: Bug 1979830 - Part 10: Age-gated eviction of live isolated arenas' mIsolatedExtents r?pbone → Bug 1979830 - Part 10: Age-gated eviction of live isolated arenas' mIsolatedExtents r?pbone
Attachment #9591207 - Attachment description: WIP: Bug 1979830 - Part 11: Mark the default arena as isolated r?pbone → Bug 1979830 - Part 11: Mark the default arena as isolated r?pbone
Depends on: 2043962

Comment on attachment 9574693 [details]
Bug 1979830 - Part 2: Replace arena_t::mSpare with mSpareChunks list r?pbone

Revision D297082 was moved to bug 2043962. Setting attachment 9574693 [details] to obsolete.

Attachment #9574693 - Attachment is obsolete: true

Comment on attachment 9574694 [details]
Bug 1979830 - Part 3: Configurable spare-chunk retention floor and Purge drain preamble r?pbone

Revision D297083 was moved to bug 2043962. Setting attachment 9574694 [details] to obsolete.

Attachment #9574694 - Attachment is obsolete: true

Comment on attachment 9574695 [details]
Bug 1979830 - Part 4: Opt high-churn arenas into 4 retained spares r?pbone

Revision D297084 was moved to bug 2043962. Setting attachment 9574695 [details] to obsolete.

Attachment #9574695 - Attachment is obsolete: true
Attachment #9574696 - Attachment description: Bug 1979830 - Part 5: Refactor chunk recycling into ExtentPool, drop the global instance r?pbone → WIP: Bug 1979830 - Part 1: Add ExtentPool class (no callers yet) r?pbone
Attachment #9574697 - Attachment description: Bug 1979830 - Part 6: Add isolated-arena mechanism (no callers yet) r?pbone → WIP: Bug 1979830 - Part 2: Add isolated-arena mechanism (no callers yet) r?pbone
Attachment #9574698 - Attachment description: Bug 1979830 - Part 7: Mark JS arenas as isolated r?pbone → WIP: Bug 1979830 - Part 3: Mark JS arenas as isolated r?pbone
Attachment #9591204 - Attachment description: Bug 1979830 - Part 8: Age-gated graveyard eviction via MayPurgeSteps r?pbone → WIP: Bug 1979830 - Part 4: Age-gated graveyard eviction via MayPurgeSteps r?pbone
Attachment #9591205 - Attachment description: Bug 1979830 - Part 9: Mark DOM arenas as isolated r?pbone → WIP: Bug 1979830 - Part 5: Mark DOM arenas as isolated r?pbone
Attachment #9591206 - Attachment description: Bug 1979830 - Part 10: Age-gated eviction of live isolated arenas' mIsolatedExtents r?pbone → WIP: Bug 1979830 - Part 6: Age-gated eviction of live isolated arenas' mIsolatedExtents r?pbone
Attachment #9591207 - Attachment description: Bug 1979830 - Part 11: Mark the default arena as isolated r?pbone → WIP: Bug 1979830 - Part 7: Mark the default arena as isolated r?pbone
Attachment #9574692 - Attachment is obsolete: true
See Also: → 2009188
Depends on: 2055507
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: