1683284 - Crash in [@ OOM | large | xul.dll | BaseThreadInitThunk ] (parent crash)

Status: NEW

(Core :: Graphics: WebRender, defect, P3)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

All of these frequent crashes are for x86 and almost all on Windows 7. They all seem to hit when the user views PDFs.

The same crash signature had been observed for 84.0b3 and no other 84 betas. Is it a regression from bug 1677818?

• 84.0b3 changes
• 84.0b4 changes

Crash report: https://crash-stats.mozilla.org/report/index/3e6f7598-149e-4057-b72d-a28f40201218

MOZ_CRASH Reason: MOZ_CRASH()

Top 10 frames of crashing thread:

0 mozglue.dll mozalloc_abort memory/mozalloc/mozalloc_abort.cpp:33
1 mozglue.dll mozalloc_handle_oom memory/mozalloc/mozalloc_oom.cpp:51
2 xul.dll xul.dll@0x36a40aa 
3 xul.dll xul.dll@0x6df638 
4 xul.dll xul.dll@0x4349b4e 
5 xul.dll xul.dll@0x4342725 
6 xul.dll xul.dll@0x4341e76 
7 xul.dll xul.dll@0x38cfa67 
8 xul.dll xul.dll@0x143bc18 
9 xul.dll xul.dll@0x143b0a3 

Comment 1

[Jamie Nicol [:jnicol]]

All the crash reports have lots of graphics critical errors, some about the shader cache and some about ShmemTextureHost failures.

Comment 2

[Julien Cristau [:jcristau] (back April 22)]

Lots of those on 85 beta...

Comment 3

[Julien Cristau [:jcristau] (back April 22)]

Jim can we get this looked into for 85?

Comment 4

[Jeff Muizelaar [:jrmuizel]]

I wonder if this bug is a signature change. None of the crashes seem to be properly symbolicated. We just get xul.dll.

Comment 5

[Gabriele Svelto [:gsvelto]]

This is most likely a signature change. The minidump is missing xul.dll's debug ID so we can't symbolcate it properly, it tends to happen on Windows during catastrophic OOMs and these definitely are. For the rest the stacks are extremely similar to other OOMs where we don't have the xul.dll debug ID.

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

40-50% of the crashes with [@ xul.dll | _PR_NativeRunThread | pr_root] have a .pdf url and all for the last month are on Windows 7 x86. The signature started to spike on beta starting with Gecko 86 (signature change from [@ OOM | small] ?). Both signatures are tracked by bug 1519123.

Comment 7

[Pascal Chevrel:pascalc (PTO until April 26)]

Jim, I am worried by the increase in volume in 86 beta vs 85 beta. Could we have somebody look at this crasher?
Also, it does not seem to be limited to windows 7 only, 69% of crashes for xul.dll | _PR_NativeRunThread | pr_root are on Windows 10. Thanks

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Maybe this increase was caused by enabling software webrender on beta 86? Software webrender will not ride into release in 86 so perhaps we'll see a drop when early beta ends.

This bug is not very easy to make actionable as basically all we know is that it's an OOM large.

Comment 9

[Julien Cristau [:jcristau] (back April 22)]

The volume looks like it did drop a little this week from 86.0b7 (late beta).

Comment 10

[Pascal Chevrel:pascalc (PTO until April 26)]

The level of crashes is similar on 86 late betas as they were on 85 betas so I think we are not in a worse shape as we were for 85.

Comment 11

[Julien Cristau [:jcristau] (back April 22)]

Unsurprisingly this is spiking up again as sw-wr is enabled in early beta 87.

Comment 12

[Jim Mathies [:jimm]]

Keeping this in triage for now, we're doing some work in 88 for x86.

Comment 13

[Andrew Osmond [:aosmond] (he/him)]

Bug 1699224 may help if we can't find the virtual memory space (physical memory is low too).

Comment 14

[Jim Mathies [:jimm]]

Related - Bug 1695379 - WebRender ANGLE Crash in [@ OOM | large | mozalloc_abort | moz_xmalloc | std::vector<T>::_Emplace_reallocate<T>]

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Bug 1699224 doesn't seem like something we can/should backport to Beta, so calling this wontfix for 88. Hopefully things improve in 89 :(

Comment 16

[Jim Mathies [:jimm]]

Waiting on 89 beta to see how much things have improved. We've landed changes there to help. Also, swiggle will not ship to 32-bit in 88.

Comment 17

[Wayne Mery (:wsmwk)]

(In reply to Jim Mathies [:jimm] from comment #16)

Waiting on 89 beta to see how much things have improved. We've landed changes there to help. Also, swiggle will not ship to 32-bit in 88.

Firefox 89 crashes cut by ~70%
https://crash-stats.mozilla.org/signature/?product=Firefox&signature=xul.dll%20%7C%20_PR_NativeRunThread%20%7C%20pr_root&date=%3E%3D2021-04-04T04%3A15%3A00.000Z&date=%3C2021-10-04T04%3A15%3A00.000Z#graphs

Thunderbird crash rate unchanged, but perhaps unrelated

No corresponding changes for https://crash-stats.mozilla.org/signature/?signature=OOM%20%7C%20large%20%7C%20mozalloc_abort%20%7C%20xul.dll%20%7C%20BaseThreadInitThunk&date=%3E%3D2021-04-04T04%3A15%3A00.000Z&date=%3C2021-10-04T04%3A15%3A00.000Z#graphs

Comment 18

[Jim Mathies [:jimm]]

I wonder why the stacks are not symbolicated for this signature? Something related to this being an OOM?

https://crash-stats.mozilla.org/signature/?signature=OOM%20%7C%20large%20%7C%20mozalloc_abort%20%7C%20xul.dll%20%7C%20BaseThreadInitThunk&date=%3E%3D2021-10-21T18%3A22%3A00.000Z&date=%3C2021-10-28T18%3A22%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_columns=startup_crash&_sort=-date&page=1#reports

Comment 19

[Gabriele Svelto [:gsvelto]]

(In reply to Jim Mathies [:jimm] from comment #18)

I wonder why the stacks are not symbolicated for this signature? Something related to this being an OOM?

https://crash-stats.mozilla.org/signature/?signature=OOM%20%7C%20large%20%7C%20mozalloc_abort%20%7C%20xul.dll%20%7C%20BaseThreadInitThunk&date=%3E%3D2021-10-21T18%3A22%3A00.000Z&date=%3C2021-10-28T18%3A22%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_columns=startup_crash&_sort=-date&page=1#reports

Yes, if you check the modules tab in one of the crashes you'll see that the xul.dll entry has no debug identifier because it was unloaded - probably because of a catastrophic OOM scenario. I hope we'll be able to symbolicate those kind of crashes next year, we're thinking about ways of working around this particular issue.

Comment 20

[GerardoPcp04]

New Crash with Windows 7 x86 and Firefox 97.0.1:
https://crash-stats.mozilla.org/report/index/6b5f59e2-3a9a-40dd-9695-1138e0220301#tab-bugzilla

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criteria:

• Top 20 desktop browser crashes on release
• Top 20 desktop browser crashes on beta
• Top 5 desktop browser crashes on Windows on release

:gw, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 22

[Glenn Watson [:gw]]

Seems like a generic OOM.

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criteria:

• Top 20 desktop browser crashes on release (startup)
• Top 20 desktop browser crashes on beta
• Top 5 desktop browser crashes on Windows on release (startup)

:gw, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.