Closed Bug 1825290 Opened 3 years ago Closed 2 years ago

Windows mozilla::mscom::ProcessRuntime::InitInsideApartment Utility crash

Categories

(Core :: IPC: MSCOM, defect)

Product:
Core
Component:
IPC: MSCOM
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: gerard-majax, Unassigned)

References

Details

Crash Data

With the work in bug 1824080 we are starting to get some telemetry data showing audio decoding happening on the tab process when utility pref is not disabled: this is not something we like.

In https://sql.telemetry.mozilla.org/queries/91032/source we can link between client_id reporting audio decoding on another process than utility and not having disabled utility, and this yields a few records so far (telemetry is in place since less than a week).

The stacks that seems to come out of those are like:

$ fx-crash-sig --verbose < crashping.json
Symbolicated stack:
   0    mozilla::mscom::ProcessRuntime::InitInsideApartment()
   1    mozilla::mscom::ProcessRuntime::ProcessRuntime(const mozilla::mscom::ProcessRuntime::ProcessCategory)
   2    XRE_InitChildProcess(int, char**, XREChildData const*)
   3    wmain(int, wchar_t**)
   4    __scrt_common_main_seh()
   5    BaseThreadInitThunk
   6    RtlUserThreadStart
signature: mozilla::mscom::ProcessRuntime::InitInsideApartment
notes:     (0)

Searching for that signature, while a bit generic, yields results, e.g., https://crash-stats.mozilla.org/report/index/cd92ac64-01bf-42ae-89c3-7c95b0230308

I believe we might have something blocking proper loading of Utility that might explain at least in some cases why we end up doing audio decoding on non utility process

Component: Other → IPC
Product: External Software Affecting Firefox → Core
Severity: -- → S2

Some other are starting to manifest:

Symbolicated stack:
   0    RaiseException
   1    __delayLoadHelper2(ImgDelayDescr const*, long long (**)())
   2    _tailMerge_oleaut32.dll
   3    mozilla::mscom::ProcessRuntime::InitInsideApartment()
   4    mozilla::mscom::ProcessRuntime::ProcessRuntime(const mozilla::mscom::ProcessRuntime::ProcessCategory)
   5    XRE_InitChildProcess(int, char**, XREChildData const*)
   6    wmain(int, wchar_t**)
   7    __scrt_common_main_seh()
   8    BaseThreadInitThunk
   9    RtlUserThreadStart
signature: __delayLoadHelper2 | _tailMerge_oleaut32.dll | mozilla::mscom::ProcessRuntime::InitInsideApartment
notes:     (0)
Component: IPC → IPC: MSCOM
Crash Signature: [@ mozilla::mscom::ProcessRuntime::InitInsideApartment]
Crash Signature: [@ mozilla::mscom::ProcessRuntime::InitInsideApartment] → [@ mozilla::mscom::ProcessRuntime::InitInsideApartment] [@ __delayLoadHelper2 | _tailMerge_oleaut32.dll | mozilla::mscom::ProcessRuntime::ProcessRuntime]

I am starting to wonder how much this is related to bug 1827379: could our failure to load oleaut32 as reported here be related to the failure to preload mozavcodec.dll and mozavutil.dll ?

Since the crash volume is low (less than 15 per week), the severity is downgraded to S3. Feel free to change it back if you think the bug is still critical.

For more information, please visit BugBot documentation.

Severity: S2 → S3

I guess the bot doesn't know about crash ping telemetry :-/

Severity: S3 → S2

We're getting some crash reports now and I see SbieDll.dll loaded which seems to be https://github.com/sandboxie-plus/Sandboxie/

Ok, local procmon I see we delay load oleaut32.dll from SetOaNoCache() https://searchfox.org/mozilla-central/rev/962a843f6d96283c45162c788dc72bf217fca7df/ipc/mscom/ProcessRuntime.cpp#144,293

Depends on: 1839463

The bug is linked to a topcrash signature, which matches the following criteria:

  • Top 20 desktop browser crashes on beta
  • Top 10 content process crashes on beta

For more information, please visit BugBot documentation.

Keywords: topcrash

We seem to have no more report on utility now that bug 1839463 has landed. Let's verify this in a few weeks.

Flags: needinfo?(lissyx+mozillians)
Flags: needinfo?(lissyx+mozillians)
Summary: Windows mozilla::mscom::ProcessRuntime::InitInsideApartment Utility crash leading to content process decoding → Windows mozilla::mscom::ProcessRuntime::InitInsideApartment Utility crash potentially to content process decoding

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.

Keywords: topcrash

(In reply to :gerard-majax [PTO 01/08/2023-10/09/2023] from comment #8)

We seem to have no more report on utility now that bug 1839463 has landed. Let's verify this in a few weeks.

It looks like we have no more report on beta for the past month.

During nightly crash triage I found two more signatures that could be relevant to this bug:

Only the first one is flagged as an OOM, but the second one could as well be one. Also several crashes under the main signature here are from 32-bit machines with very little address space left, so they could be OOMs too.

Sorry for removing the keyword earlier but there is a recent change in the ranking, so the bug is again linked to a topcrash signature, which matches the following criteria:

  • Top 20 desktop browser crashes on beta
  • Top 10 content process crashes on beta

For more information, please visit BugBot documentation.

Keywords: topcrash

(In reply to Gabriele Svelto [:gsvelto] from comment #11)

During nightly crash triage I found two more signatures that could be relevant to this bug:

Only the first one is flagged as an OOM, but the second one could as well be one. Also several crashes under the main signature here are from 32-bit machines with very little address space left, so they could be OOMs too.

Looking at those crashes, they only get reported on Parent and Content processes, not utility

Based on the topcrash criteria, the crash signatures linked to this bug are not in the topcrash signatures anymore.

For more information, please visit BugBot documentation.

Keywords: topcrash

Crash Ping Telemetry signals 14 clients on release, lowering severity.

Severity: S2 → S3

It seems we got rid of all of them for Utility: crash-stats reports only Content processes on this signatures.

We know crash-stats might not be ultra reliable for non content/parent processes, and looking at crash reports from telemetry it also looks to have either disappeared or be so low volume it's not impacting:

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WORKSFORME
Summary: Windows mozilla::mscom::ProcessRuntime::InitInsideApartment Utility crash potentially to content process decoding → Windows mozilla::mscom::ProcessRuntime::InitInsideApartment Utility crash
You need to log in before you can comment on or make changes to this bug.