1699224 - Support remapping shared surfaces when there is virtual memory pressure

Status: RESOLVED FIXED

(Core :: Graphics: WebRender, defect, P3)

Description

[Andrew Osmond [:aosmond] (he/him)]

For 32-bit users, primarily Windows and Android, encounter OOMs due to low virtual memory address space a lot. If we could unmap the largest least recently used images when we hit memory pressure, and remap them on demand, that would likely help a lot. This would be similar to the volatile memory efforts without WebRender, but without relying upon an OS mechanism to do it.

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1699224 - Part 1. Ensure we map/unmap surfaces used by DrawTargetSkia.

Since we want to be able to unmap shared surfaces which are not in use,
we need to be able to map the data in as required. Blob images can
access shared surfaces via recordings which are used by DrawTargetSkia.
This patch makes it call Map/Unmap for data surfaces that require a data
pointer in Skia.

Comment 2

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1699224 - Part 2. Expose SharedMemoryBasic::Unmap for use by callers.

This patch exposes SharedMemoryBasic::Unmap for use by callers to allow
one to unmap memory out of our address space without destroying the
SharedMemoryBasic object itself. This will be used by later patches in
this series.

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1699224 - Part 3. Complete shutdown of SharedSurfacesParent on the main thread.

We want to use nsExpirationTracker to decide when we should unmap shared
surfaces from our address space. This relies upon the observer service
and we need to be both initialized and shutdown on the main thread to
add/remove the observer. SharedSurfacesParent is already initialized on
the main thread, and we just need to complete shutdown on the main
thread.

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1699224 - Part 4. Allow shared surfaces to be mapped and unmapped on demand.

This patch adds an expiration tracker to decide when to unmap unused
shared surfaces from our address space to reclaim virtual memory. This
is only used on 32-bit builds of Firefox where there is meaningful
virtual address space pressure.

Comment 5

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1699224 - Part 5. Unmap shared surfaces if memory pressured during resource updates.

Before we start a resource update, we should check if we are virtual
memory pressured (32-bit Windows only). If so, pre-emptively unmap
shared surfaces until the pressure is relieved to try to avoid OOMs
elsewhere. This only applies to the GPU process because the parent
process actively watches its own memory pressure and dispatches a
low-memory event which our expiration tracker is an observer for.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f8b232382361
Part 1. Ensure we map/unmap surfaces used by DrawTargetSkia. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/8ce6ddf6ec6f
Part 2. Expose SharedMemoryBasic::Unmap for use by callers. r=jld
https://hg.mozilla.org/integration/autoland/rev/85f3499080db
Part 3. Complete shutdown of SharedSurfacesParent on the main thread. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/36bf68e01a50
Part 4. Allow shared surfaces to be mapped and unmapped on demand. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/f929f9783e61
Part 5. Unmap shared surfaces if memory pressured during resource updates. r=jrmuizel

Comment 7

[Atila Butkovits]

Backed out for causing build bustages on SharedSurfacesParent.cpp.

Backout link: https://hg.mozilla.org/integration/autoland/rev/c8f281ec530b2264517fb231155ae339fb96e230

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=f929f9783e61d5136e3a7fa49355fad58f50dbb8&selectedTaskRun=X0mCuaUxSXurf30MXj3Iww.0&searchStr=Linux%2Cx64%2Copt%2Cbuild-linux64%2Fopt%2CB

Failure log: https://treeherder.mozilla.org/logviewer?job_id=334569059&repo=autoland&lineNumber=37183

Comment 8

[Andrew Osmond [:aosmond] (he/him)]

Ugh, merge conflict error.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d2294fe6ef31
Part 1. Ensure we map/unmap surfaces used by DrawTargetSkia. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/ef3ef367986e
Part 2. Expose SharedMemoryBasic::Unmap for use by callers. r=jld
https://hg.mozilla.org/integration/autoland/rev/206187cafb6f
Part 3. Complete shutdown of SharedSurfacesParent on the main thread. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/94c75dba6545
Part 4. Allow shared surfaces to be mapped and unmapped on demand. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/bc232de0c5f8
Part 5. Unmap shared surfaces if memory pressured during resource updates. r=jrmuizel

Comment 10

[Noemi Erli[:noemi_erli]]

Backed out 5 changesets (Bug 1699224) for causing web platform and reftest failures CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=bc232de0c5f8359a6e55aef8511af61f32414330&selectedTaskRun=bwcMW8rhSLCTlhNel02vaA.0

Log: https://treeherder.mozilla.org/logviewer?job_id=334593380&repo=autoland&lineNumber=3258
https://treeherder.mozilla.org/logviewer?job_id=334592075&repo=autoland&lineNumber=4791
https://treeherder.mozilla.org/logviewer?job_id=334591384&repo=autoland&lineNumber=5534
https://treeherder.mozilla.org/logviewer?job_id=334594189&repo=autoland&lineNumber=3850
https://treeherder.mozilla.org/logviewer?job_id=334593578&repo=autoland&lineNumber=4929
https://treeherder.mozilla.org/logviewer?job_id=334593441&repo=autoland&lineNumber=1733

Backout: https://hg.mozilla.org/integration/autoland/rev/ad6d764bc67107103d3dcafffa3c8cc532ac4748

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/66b86bed3d06
Part 1. Ensure we map/unmap surfaces used by DrawTargetSkia. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/9aac5a079130
Part 2. Expose SharedMemoryBasic::Unmap for use by callers. r=jld
https://hg.mozilla.org/integration/autoland/rev/db4b82029686
Part 3. Complete shutdown of SharedSurfacesParent on the main thread. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/cd7cfc8fc140
Part 4. Allow shared surfaces to be mapped and unmapped on demand. r=jrmuizel
https://hg.mozilla.org/integration/autoland/rev/1646c323910d
Part 5. Unmap shared surfaces if memory pressured during resource updates. r=jrmuizel

Comment 12

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/66b86bed3d06
https://hg.mozilla.org/mozilla-central/rev/9aac5a079130
https://hg.mozilla.org/mozilla-central/rev/db4b82029686
https://hg.mozilla.org/mozilla-central/rev/cd7cfc8fc140
https://hg.mozilla.org/mozilla-central/rev/1646c323910d