1571516 - Startup Crash in [@ nsWidgetWindowsModuleCtor] via mozilla::xpcom::CreateInstanceImpl. MOZ_CRASH Reason MOZ_RELEASE_ASSERT(((bool)(__builtin_expect

Status: ASSIGNED

(Core :: Widget: Win32, defect, P3)

Description

[Marcia Knous [:marcia]]

This bug is for crash report bp-b3a0afe9-0107-490d-8498-2e1800190805.

Seen while looking at nightly crash stats. This crash seems to have started in 69.0b8: https://bit.ly/2Yt6L4A

All of the crash have the same Moz Crash Reason: MOZ_RELEASE_ASSERT(((bool)(__builtin_expect(!!(!NS_FAILED_impl(rv)), 1)))). Strong correlation to one CPU:

(90.91% in signature vs 00.18% overall) CPU Info = AuthenticAMD family 21 model 112 stepping 0 [100.0% vs 00.31% if cpu_arch = x86]

Full 69.0b8 changelog: https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FIREFOX_69_0b7_RELEASE&tochange=53bb71da50e59d953cdcd68a29c5afabd160abfc&full=1

Top 10 frames of crashing thread:

0 xul.dll nsWidgetWindowsModuleCtor widget/windows/nsWidgetFactory.cpp:51
1 xul.dll mozilla::xpcom::CreateInstanceImpl xpcom/components/StaticComponents.cpp:11135
2 xul.dll nsComponentManagerImpl::GetServiceLocked xpcom/components/nsComponentManager.cpp:1384
3 xul.dll nsComponentManagerImpl::GetService xpcom/components/nsComponentManager.cpp:1436
4 xul.dll nsGetServiceByCIDWithError::operator xpcom/components/nsComponentManagerUtils.cpp:230
5 xul.dll nsCOMPtr_base::assign_from_gs_cid_with_error xpcom/base/nsCOMPtr.cpp:73
6 xul.dll nsAppStartup::Init toolkit/components/startup/nsAppStartup.cpp:170
7 xul.dll mozilla::xpcom::CreateInstanceImpl xpcom/components/StaticComponents.cpp:9390
8 xul.dll nsComponentManagerImpl::GetServiceLocked xpcom/components/nsComponentManager.cpp:1384
9 xul.dll nsComponentManagerImpl::GetService xpcom/components/nsComponentManager.cpp:1474

Comment 1

[[:philipp]]

this is turning into a mid-low volume startup crash after 69 hit the release channel.

Comment 2

[(Away)]

Kats mentioned in chat that this has been appearing in telemetry pings pretty often. This is the release assert added by bug 1545381.

By inspection the failure has to be either CreateWindowW or nsIThreadInternal. I think it's the former.

I looked at what few of these we have on the nightly channel (where the crash reporter saves GetLastError information) and they say things like

0:000> !gle
LastErrorValue: (Win32) 0x57f (1407) - Cannot find window class.
LastStatusValue: (NTSTATUS) 0xc0000017 - {Not Enough Quota}  Not enough virtual memory or paging file quota is available to complete the specified operation.

or

0:000> !gle
LastErrorValue: (Win32) 0x486 (1158) - The current process has used all of its system allowance of handles for Window Manager objects.
LastStatusValue: (NTSTATUS) 0xc0000034 - Object Name not found.

Are we running up against Windows resource limits or something?

Comment 3

[Kartikaya Gupta (email:[e-mail])]

(In reply to :dmajor from comment #2)

Kats mentioned in chat that this has been appearing in telemetry pings pretty often.

From the pings it looked like it was happening an order of magnitude more frequently than any other crash. But actually it looks like pings are duplicated, in many cases there are 45 pings with the same minidump_sha256_hash value. :gsvelto said this was a previously-occurring issue that went away, but I filed bug 1660542 to track this problem. I don't yet know if there is a correlation between this crash and the repeated-pings (i.e. maybe all the repeated pings are for this nsWidgetWindowsModuleCtor crash) - I'll try to find out.

Comment 4

[Wayne Mery (:wsmwk)]

#75 crash for Thunderbird 78.12.0. bp-cab19ac6-47d9-471e-bfdd-3661f0210801 is an example.
Firefox crash rate is, surprisingly, higher. #59 for version 90.0.2.

(In reply to Marcia Knous [:marcia] from comment #0)

This bug is for crash report bp-b3a0afe9-0107-490d-8498-2e1800190805.

Seen while looking at nightly crash stats. This crash seems to have started in 69.0b8: https://bit.ly/2Yt6L4A

69.0b8 was created 26-Jul-2019

The oldest Thunderbird crash I find in the past 6 months is bp-3653093f-9274-44b4-8ce3-19ea40210222 68.6.0 buildid 20200310192757

Comment 5

[Wayne Mery (:wsmwk)]

(In reply to Kartikaya Gupta (email:[e-mail]) from comment #3)

(In reply to :dmajor from comment #2)
...
From the pings it looked like it was happening an order of magnitude more frequently than any other crash. But actually it looks like pings are duplicated, in many cases there are 45 pings with the same minidump_sha256_hash value. :gsvelto said this was a previously-occurring issue that went away, but I filed bug 1660542 to track this problem. I don't yet know if there is a correlation between this crash and the repeated-pings (i.e. maybe all the repeated pings are for this nsWidgetWindowsModuleCtor crash) - I'll try to find out.

Kats, had you come to any determination?

Comment 6

[Wayne Mery (:wsmwk)]

Also noteworthy, a noticeable decrease in crash rate occurred in mid-April https://crash-stats.mozilla.org/signature/?signature=nsWidgetWindowsModuleCtor&date=%3E%3D2021-03-28T13%3A11%3A00.000Z&date=%3C2021-09-28T13%3A11%3A00.000Z#graphs

The high water mark at that was Firefox 87.0 which had ~700 crashes over a few weeks. https://crash-stats.mozilla.org/signature/?product=Firefox&signature=nsWidgetWindowsModuleCtor&date=%3E%3D2021-03-28T00%3A00%3A00.000Z&date=%3C2021-09-28T23%3A59%3A00.000Z#summary

It's unclear whether going from 86 there was then a spike in 87, or whether 87 was the end of a sustained high rate.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

(In reply to Wayne Mery (:wsmwk) from comment #5)

Kats, had you come to any determination?

I don't recall in my head, but based on what I wrote at https://bugzilla.mozilla.org/show_bug.cgi?id=1660542#c2 it looks like there were a number of different signatures in the repeated pings, but the majority where for this crash.

Comment 8

[Chris Peterson [:cpeterson]]

I see a big increase in crash pings with crash reason MOZ_RELEASE_ASSERT(((bool)(__builtin_expect(!!(!NS_FAILED_impl(rv)), 1)))) in Firefox 94 compared to 93 from Windows 10 (but not Windows 7). Looks like Firefox 93 was the outlier (with a lower than usual crash volume). I see Firefox 89–92 had a high crash volume for the MOZ_RELEASE_ASSERT reason, similar to 94's volume. So maybe there is no regression in Firefox 94, just a return to normal after 93?

Number of crash pings with crash reason MOZ_RELEASE_ASSERT(((bool)(__builtin_expect(!!(!NS_FAILED_impl(rv)), 1)))) according to this SQL query:

https://sql.telemetry.mozilla.org/queries/82915/source

• Firefox 89 = 2.5M
• Firefox 90 = 1.5M
• Firefox 91 = 2.1M
• Firefox 92 = 1.3M
• Firefox 93 = only 400K?!
• Firefox 94 = 766K after only two weeks, almost 2x Firefox 93's volume in only half the time.

Comment 9

[Wayne Mery (:wsmwk)]

roughly 30% increase of Firefox crashes since fall.
bp-92920bc1-7381-4973-a3aa-342160211217 Firefox 95.0

bp-28915ab0-032c-482c-8312-62f310211109 uptime 0 seconds Thunderbird

0 xul.dll nsWidgetWindowsModuleCtor() widget/windows/nsWidgetFactory.cpp:49 context
1 xul.dll mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) xpcom/components/StaticComponents.cpp:12557 cfi
2 xul.dll mozilla::xpcom::StaticModule::CreateInstance(nsISupports*, nsID const&, void**) const xpcom/components/StaticComponents.cpp:14318 cfi
3 xul.dll nsComponentManagerImpl::GetServiceLocked(mozilla::Maybe<mozilla::MonitorAutoLock>&, `anonymous namespace'::EntryWrapper&, nsID const&, void**) xpcom/components/nsComponentManager.cpp:1276 cfi
4 xul.dll nsComponentManagerImpl::GetService(nsID const&, nsID const&, void**) xpcom/components/nsComponentManager.cpp:1330 cfi
5 xul.dll nsGetServiceByCIDWithError::operator()(nsID const&, void**) const xpcom/components/nsComponentManagerUtils.cpp:230 cfi
6 xul.dll nsCOMPtr_base::assign_from_gs_cid_with_error(nsGetServiceByCIDWithError const&, nsID const&) xpcom/base/nsCOMPtr.cpp:73 cfi

Comment 10

[Christian Holler (:decoder)]

Attachment: Bug 1571516 - Add diagnostics for startup crash. r?gsvelto

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0541f3affa02
Add diagnostics for startup crash. r=gsvelto

Comment 12

[Christian Holler (:decoder)]

Adding some diagnostic asserts to assess some theories here:

1. It is very likely the CreateWindowW failing here.

2. The CreateWindowW is failing with ERROR_CANNOT_FIND_WND_CLASS, so maybe the RegisterClassW before it failed.

We are now looking for either of these two diagnostic asserts on Nightly:

MOZ_DIAGNOSTIC_ASSERT(wcA, "RegisterClassW for EventWindowClass failed");

or

MOZ_DIAGNOSTIC_ASSERT(mEventWnd, "CreateWindowW for EventWindow failed");

If we see the first, then :gsvelto suggested we could be running out of ATOMs (should be indicated by ERROR_NOT_ENOUGH_MEMORY as last error). If we see the second error, then there must be some other cause for the class to be not found.

If we don't see either of them, but we see the release assert instead on Nightly, then it must be a failure in nsIThreadInternal, which is unlikely but possible.

Comment 13

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/0541f3affa02

Comment 14

[Pascal Chevrel:pascalc (PTO until April 26)]

Christian, any progress on this bug?

Comment 15

[Christian Holler (:decoder)]

Yes, we have apparently two reports for the second MOZ_DIAGNOSTIC_ASSERT, and none for the first.

The crash I am looking at is https://crash-stats.mozilla.org/report/index/f7136397-67dd-4d09-8246-50feb0220204

Gabriele, any idea what might cause CreateWindowW to fail like this? Also, "Last Error Value" is "ERROR_SUCCESS" (?).

Comment 16

[Christian Holler (:decoder)]

Looking closer, I think the reason why CreateWindowW returns NULL here without failing is that one of the two early returns in this function is taken?

https://searchfox.org/mozilla-central/rev/68a5327697ec43aa55b458c504c4b313c9c80528/widget/windows/nsAppShell.cpp#351

Comment 17

[Gabriele Svelto [:gsvelto] (PTO)]

Looking at recent crash reports we've just received a burst for reports from the first exception in beta. They all have very short uptimes and are all startup crashes. The two crashes with the second assertion are different.

(In reply to Christian Holler (:decoder) from comment #16)

Looking closer, I think the reason why CreateWindowW returns NULL here without failing is that one of the two early returns in this function is taken?

https://searchfox.org/mozilla-central/rev/68a5327697ec43aa55b458c504c4b313c9c80528/widget/windows/nsAppShell.cpp#351

It sounds possible but I'm not familiar with how CreateWindowW interacts with that procedure; that is I don't know what it's supposed to return and how CreateWindowW would respond to it. There's a few hints about it in the documentation, see this:

Before returning, CreateWindow sends a WM_CREATE message to the window procedure. For overlapped, pop-up, and child windows, CreateWindow sends WM_CREATE, WM_GETMINMAXINFO, and WM_NCCREATE messages to the window. The lParam parameter of the WM_CREATE message contains a pointer to a CREATESTRUCT structure. If the WS_VISIBLE style is specified, CreateWindow sends the window all the messages required to activate and show the window.

Note how it doesn't mention that the function might return a failure. We might have to explicitly call GetLastError() to figure out more. There's another thing to keep in mind: both crashes have large UptimeTS values and are in content processes, so they happened while Firefox had been running for a while.

Which brings me to my second consideration: the new crashes we're receiving are in beta and don't have the last error value set because it's not captured in the minidumps; only nightly captures that information. So while we know that in those cases it was RegisterClassW() that failed but we don't know why.

So we're dealing with two significantly different issues here and we still don't have enough information to figure out what's going on :(

Comment 18

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:decoder, maybe it's time to close this bug?
For more information, please visit auto_nag documentation.

Comment 19

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signature linked to this bug is not a topcrash signature anymore.

For more information, please visit auto_nag documentation.

Comment 21

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 22

[BugBot [:suhaib / :marco/ :calixte]]

The bug is linked to a topcrash signature, which matches the following criterion:

• Top 20 desktop browser crashes on release (startup)

:decoder, could you consider increasing the severity of this top-crash bug?

For more information, please visit auto_nag documentation.

Comment 23

[Christian Holler (:decoder)]

Based on comment 17, I guess we should extend the diagnostic to include GetLastError() data? (I thought we had this already in crash dumps).

Also, the duplicates might not actually be duplicates, as comment 17 suggests, we might be looking at two (potentially unrelated) failures.

Comment 24

[Gabriele Svelto [:gsvelto] (PTO)]

(In reply to Christian Holler (:decoder) from comment #23)

Based on comment 17, I guess we should extend the diagnostic to include GetLastError() data? (I thought we had this already in crash dumps).

We do, but only for nightly where we capture more data than on release/beta. This is something we could extend to beta but it would require more storage on crash-stats as the minidumps would roughly double in size.

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit auto_nag documentation.

Comment 26

[Wayne Mery (:wsmwk)]

FWIW, FIrefox crashes die out for nsWidgetWindowsModuleCtor at Firefox 105.0.3 https://crash-stats.mozilla.org/signature/?signature=nsWidgetWindowsModuleCtor&date=%3E%3D2022-09-17T13%3A18%3A00.000Z&date=%3C2022-12-17T13%3A18%3A00.000Z#summary

https://crash-stats.mozilla.org/signature/?signature=nsAppShell%3A%3AInit&date=%3E%3D2022-09-17T13%3A17%3A00.000Z&date=%3C2022-12-17T13%3A17%3A00.000Z#summary shows up only for nightlies

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

Copying crash signatures from duplicate bugs.

Comment 29

[BugBot [:suhaib / :marco/ :calixte]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly

For more information, please visit BugBot documentation.

Comment 30

[BugBot [:suhaib / :marco/ :calixte]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criterion:

• Top 10 desktop browser crashes on nightly (startup)

For more information, please visit BugBot documentation.

Comment 31

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.

Comment 32

[BugBot [:suhaib / :marco/ :calixte]]

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to topcrash signatures, which match the following criteria:

• Top 10 desktop browser crashes on nightly
• Top 20 desktop browser crashes on release (startup)

For more information, please visit BugBot documentation.

Comment 33

[Ray Kraesig [:rkraesig]]

Attachment: Bug 1571516 - [1/2] preparatory reorganization r?gsvelto,bobowen

Pull the failing section of code out into its own function, for the sake
of clarity of later diffs.

No functional changes.

Comment 34

[Ray Kraesig [:rkraesig]]

Attachment: Bug 1571516 - [2/2] store diagnostic data within stack frame r?gsvelto,bobowen

Per https://bugzilla.mozilla.org/show_bug.cgi?id=1571516#c24, we don't
currently store diagnostic data like the value of ::GetLastError() in
stack dumps. However, we do store the current stack, and that's enough
to be usable.

Store some additional (critical?) diagnostic information in local
variables on the stack... and disable optimizations for that function,
to ensure that those variables actually exist in the stack-dump.

Arguably, no functional changes.

Depends on D186987

Comment 35

[Ray Kraesig [:rkraesig]]

(In reply to Gabriele Svelto [:gsvelto] from comment #24)

(In reply to Christian Holler (:decoder) from comment #23)

Based on comment 17, I guess we should extend the diagnostic to include GetLastError() data? (I thought we had this already in crash dumps).

We do, but only for nightly where we capture more data than on release/beta. This is something we could extend to beta but it would require more storage on crash-stats as the minidumps would roughly double in size.

I've written a couple of commits to store the return values of GetLastError() and RtlGetLastNtStatus() on the stack for this bug, but I'm curious why this is so. Could we reasonably send up a tiny minidump-stream that has nothing but per-thread error data?

Comment 36

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0664ee61f82f
[1/2] preparatory reorganization  r=gsvelto
https://hg.mozilla.org/integration/autoland/rev/d706cd5fccfe
[2/2] store diagnostic data within stack frame  r=gsvelto

Comment 37

[Atila Butkovits]

Backed out for causing bustages at nsAppShell.

Backout link: https://hg.mozilla.org/integration/autoland/rev/309516354257bd6150737827fb4dce0824cfb371

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=d706cd5fccfead4d761722481ffff6a64a8469d0&selectedTaskRun=fMUSCvnrSSyXsloXEHs4XA.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=427510855&repo=autoland&lineNumber=73459

Comment 38

[Ray Kraesig [:rkraesig]]

Ah, non-unified builds. On it.

Comment 39

[Ray Kraesig [:rkraesig]]

Relanding. (Green Bp-nu build here.)

Comment 40

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a395e4645323
[1/2] preparatory reorganization  r=gsvelto
https://hg.mozilla.org/integration/autoland/rev/772112a9613e
[2/2] store diagnostic data within stack frame  r=gsvelto

Comment 41

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/a395e4645323
https://hg.mozilla.org/mozilla-central/rev/772112a9613e

Comment 42

[Ray Kraesig [:rkraesig]]

Well, we can rule out a few causes. ...probably.

Minidump analyses

In recent crashes from Nightly, _sAppShellGeckoMsgId is 0, indicating that the call to ::RegisterWindowMessage() is failing. The crash itself is then the result of the later ::CreateWindow() call incompletely failing (returning no HWND, but also not setting ::GetLastError()). Unfortunately, ::RegisterWindowMessage()'s failure is similarly incomplete. (::RtlGetLastNtStatus() does return 0xc0000047, STATUS_SEMAPHORE_LIMIT_EXCEEDED, but that may be cruft left over from a previous failure, rather than being from ::RegisterWindowMessage() itself.)

The obvious reason for ::RegisterWindowMessage() failing is exhaustion of the relevant global atom table on the system in question. I've deliberately induced that on my Win10 box, though, and in that case ::GetLastError() returns 8, ERROR_NOT_ENOUGH_MEMORY, and ::RtlGetLastNtStatus() returns 0xc0000017, STATUS_NO_MEMORY. I've been able to generate a few crash reports in this state (release [a], [b]; nightly [a] [b]); release builds currently don't have enough information to make a distinction, but unfortunately, the nightly crashes are completely different.

Even the assumption that ::RegisterWindowMessage() went off the rails would do little to explain the failure of ::CreateWindow() [1]. "little" is perhaps not quite nothing, however? An obscure corner of the Microsoft documentation notes that "Many critical APIs, including CreateWindow, rely on user atoms. Therefore, space exhaustion in the user atom table will result in serious issues; for example, all applications may fail to launch." (Still, I'd expect most applications' failures-to-start to involve RegisterClass failing, rather than CreateWindow.)

As implicitly suggested in comment 16 above, another reason would be if ::CreateWindow() sent in a message with ID 0 (WM_NULL), checking whether or not it was ultimately forwarded to ::DefWindowProc(). That has been observed to result in null-response without an error, but that probably isn't what's happening here: the messages customarily passed to a top-level window are, in order, [WM_GETMINMAXINFO, WM_NCCREATE, WM_NCCALCSIZE, WM_CREATE]. (There is no canonical documentation for this fact, nor likely ever will be; but experimenters have observed this at least as far back as XP SP3.)

We could prevent the ::RegisterWindowMessage() failure specifically by not trying to dynamically register a window message -- there doesn't seem to be any immediate reason we need one here; a WM_USER-based message should be fine -- but this would do nothing for the following failure of ::CreateWindow().

Statistics

CreateWindowW-based crashes occur almost exclusively in Windows 11.

RegisterClassW-based crashes seem only slightly more likely to occur in Windows 11 than Windows 10 (assuming the Windows 10/11 ratio for OOM-small crashes is a good indicator of our per-capita usage).

Comment 43

[Ray Kraesig [:rkraesig]]

Attachment: Bug 1571516 - log additional data to stack (redux) r?gsvelto,#win-reviewers

Improve logging around InitHiddenWindow():

• Normalize WinErrorState.

• Simplify access to the OS-cached last-NTSTATUS value.

• Store and clear the last error-state more frequently.

• When ::CreateWindowW() fails, keep enough information to determine
  whether a call to ::RegisterClassW() was attempted.

• To rule out user atom table exhaustion, check the capacity and
  availability of the user atom table after a call that might have
  created a user atom fails.

• To rule out early return from our WndProc, when
  ::RegisterWindowMessageW() fails, leave the extant value of
  sAppShellGeckoMsgId in place.

Comment 44

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3fcae86ead46
log additional data to stack (redux)  r=gsvelto,win-reviewers,mhowell

Comment 45

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/3fcae86ead46

Comment 46

[Yannis Juglaret [:yannis]]

Attachment: Bug 1571516 - Additional attempt at collecting debug information for CreateWindowW failures. r=rkraesig

This patch runs RegisterClassW and CreateWindowW a second time when they
fail, so that we can trace their execution and collect single-step data.
We collect RIP values as we step through user32 code. We also collect
changes in the last error value while in user32. We store the collected
data in the stack so that it appears in the crash dumps.

Because this patch only affects behavior after a first failure (which
would normally lead to an immediate crash), it does not alter behavior
for non-crashing users.

Comment 47

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8e96effae5d1
Additional attempt at collecting debug information for CreateWindowW failures. r=win-reviewers,rkraesig

Comment 48

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/8e96effae5d1

Comment 49

[Yannis Juglaret [:yannis]]

Adding the signature where crashes with single step data should pop. Example crash report from myself: here

Comment 50

[Yannis Juglaret [:yannis]]

We have received some crashes with single step data from nightly users. All nightly crashes received so far (except the crash I submitted myself) occured within a plugin process, whereas the release crashes are almost exclusively in the main process. Therefore, there is no guarantee that users in release crash for the same reason as what I will detail here, but investigating the Nightly crashes seems to be our best bet for the moment.

The nightly crashes received so far all seem to have the same root cause. In these crashes, CreateWindowW failed as per propagation of a failure of the NtUserRegisterClassExWOW win32k system call under the following call stack:

00 win32u!NtUserRegisterClassExWOW
01 user32!RegisterClassExWOWW
02 user32!RegisterIMEClass
03 user32!VerNtUserCreateWindowEx
04 user32!CreateWindowInternal
05 user32!CreateWindowExW

Creating a window requires the registration of a dedicated internal class for IME, which, if successful, occurs only once per process. Here this registration failed by returning a 0 atom, without setting a GetLastError value nor an NTSTATUS value. Note that there are other attempts at creating a window in our code base, in other paths that run before nsAppShellInit (as part of CreateAndStorePreXULSkeletonUIImpl, and also as part of calls to CoInitializeEx), and these must have failed too, otherwise the IME class would already be registered; however failure there is not fatal currently.

Because NtUserRegisterClassExWOW runs in kernel mode, we unfortunately have no single step data for it. But, we have some hints. First, the issue does not appear to be a general problem with class registration, since our own call to RegisterClassW succeeds, and that also goes through RegisterClassExWOWW (with fnid=0) and NtUserRegisterClassExWOW. Instead, it could be a problem with the registration of the IME class specifically (i.e. when RegisterClassExWOWW is called with fnid=FNID_IME), or of internal classes in general (i.e. when RegisterClassExWOWW is called with a non-zero fnid value in general). Second, most failure paths in NtUserRegisterClassExWOW from win32kfull.sys do seem to set a last error code, so we can potentially use the fact that none is set in our case to guess what may be happening. That will require additional investigation.

Comment 51

[BugBot [:suhaib / :marco/ :calixte]]

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.