1683653 - Crash in [@ OOM | large | mozalloc_abort | moz_xmalloc | mozilla::gfx::SourceSurfaceRawData::GuaranteePersistance]

Status: RESOLVED FIXED

(Core :: Graphics, defect, P2)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This looks like bug 1604535 resurrected, including mostly hitting during Facebook visits. The signature was active before but got more frequent starting with the Firefox 85 betas.

Crash report: https://crash-stats.mozilla.org/report/index/645260f9-68d4-4af7-9a17-a311d0201220

MOZ_CRASH Reason: MOZ_CRASH()

Top 10 frames of crashing thread:

0 mozglue.dll mozalloc_abort memory/mozalloc/mozalloc_abort.cpp:33
1 mozglue.dll mozalloc_handle_oom memory/mozalloc/mozalloc_oom.cpp:51
2 mozglue.dll moz_xmalloc memory/mozalloc/mozalloc.cpp:54
3 xul.dll mozilla::gfx::SourceSurfaceRawData::GuaranteePersistance gfx/2d/SourceSurfaceRawData.cpp:39
4 xul.dll mozilla::gfx::DrawTargetTiled::FillRect gfx/2d/DrawTargetTiled.cpp:229
5 xul.dll mozilla::layers::FillRectWithMask gfx/layers/basic/BasicLayersImpl.cpp:129
6 xul.dll mozilla::layers::FillRectWithMask gfx/layers/basic/BasicLayersImpl.cpp:151
7 xul.dll mozilla::layers::BasicImageLayer::Paint gfx/layers/basic/BasicImageLayer.cpp:80
8 xul.dll mozilla::layers::BasicLayerManager::PaintSelfOrChildren gfx/layers/basic/BasicLayerManager.cpp:701
9 xul.dll mozilla::layers::BasicLayerManager::PaintLayer gfx/layers/basic/BasicLayerManager.cpp:871

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

This is probably not an image -- none of the surfaces from imagelib would trigger a copy during GuaranteePersistance. This is most likely a video or canvas.

Comment 2

[Julien Cristau [:jcristau]]

Andrew can you look into this or redirect to media folks? The OOM increase between 84 and 85 is a bit worrying.

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

I'll disable OMTP for more cases. We already disable it for users with < 2 GB physical memory, and the majority of these crashes are for users with > 2 GB memory but < 2 GB virtual memory (most 32-bit Firefox on 32-bit Windows users I imagine fall into this bucket). It will take too long to debug this and this sort of allocation will not happen very often with WebRender.

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1683653 - Disable OMTP for users with less than 2 GB of virtual memory.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/65e409aa846d
Disable OMTP for users with less than 2 GB of virtual memory. r=jrmuizel

Comment 6

[Atila Butkovits]

Backed out for causing reftest failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/b9f7f8d17f62f51076bfca3cff8f0986fa091ff1

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&searchStr=windows%2C7%2Cdebug%2Creftests%2Ctest-windows7-32%2Fdebug-reftest-gpu-e10s%2Crg3&revision=65e409aa846d9b0f928e300db581b5e4d5c4ab08&selectedTaskRun=PoHQrn2eS-CTpRU0Wqjnvg.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=326104679&repo=autoland&lineNumber=25944
https://treeherder.mozilla.org/logviewer?job_id=326104679&repo=autoland&lineNumber=25944

Comment 7

[Andrew Osmond [:aosmond] (he/him)]

Added fuzz, relanding.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d4d31680d3ce
Disable OMTP for users with less than 2 GB of virtual memory. r=jrmuizel

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/d4d31680d3ce

Comment 10

[Andrew Osmond [:aosmond] (he/him)]

Comment on attachment 9195915 [details]
Bug 1683653 - Disable OMTP for users with less than 2 GB of virtual memory.

Beta/Release Uplift Approval Request

• User impact if declined: OTMP users with low powered hardware may experience more frequent OOMs
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is just an expansion of a previous patch to disable OMTP for some users. Very simple criteria. Covered by Windows 7 automated tests. Worst case we disable OMTP for more users than we expect but as we continue to migrate more users to WebRender (which doesn't rely on OMTP) this should become less of an issue.
• String changes made/needed:

Comment 11

[Andrew Osmond [:aosmond] (he/him)]

I don't think we can truly verify in nightly just because this doesn't solve the crashes, it just removes the users most likely to be affected and benefit the least from OMTP. The crash rate isn't high enough in nightly to confirm this.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9195915 [details]
Bug 1683653 - Disable OMTP for users with less than 2 GB of virtual memory.

Approved for 85.0rc1.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/a2ed4a64e27a

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Should a new bug be created for the crash reports with 86+ or shall this one be reopened?

Comment 15

[Andrew Osmond [:aosmond] (he/him)]

We disabled OMTP in bug 1694248 so they should go away in 88.

Comment 16

[Worcester12345]

memory-leak key word?