1901230 - Crash in [@ RtlpFreeHeapInternal | RtlFreeHeap | APP_DATA::FreeCachedMem] (Database.dll) - SOLIDWORKS when using the mozglue blocklist (affects Thunderbird) or out-of-process file picker

Status: NEW

(External Software Affecting Firefox :: Other, defect)

Description

[Yannis Juglaret [:yannis]]

We are receiving crashes that involve Database.dll, which belongs to SolidWorks by Dassault Système. This is a double-free heap corruption, exactly like described in meta bug 1900964. The DLL is typically located at C:\Program Files\SOLIDWORKS CorpXXXX\SOLIDWORKS PDM\Database.dll where XXXX is a year like 2018 or 2019. These crashes are the continuation of bug 1468250 and bug 1566109, which caused the DLL to be added to our DLL blocklist, though only for the main process. This DLL is still responsible for about 17.07% of the volume on the attached signature in the last six months.

Call stack:

00 ntdll!RtlReportFatalFailure+0x9
01 ntdll!RtlReportCriticalFailure+0x97
02 ntdll!RtlpHeapHandleError+0x12
03 ntdll!RtlpHpHeapHandleError+0x7a
04 ntdll!RtlpLogHeapFailure+0x45
05 ntdll!RtlpFreeHeapInternal+0x4e0
06 ntdll!RtlFreeHeap+0x51
07 oleaut32!APP_DATA::FreeCachedMem+0xbb
08 oleaut32!SysFreeString+0x4e
09 Database+0x22ad
...

The block resulting from bug 1468250 and bug 1566109 has successfully prevented a lot of crashes in Firefox Release and Beta. However we are still getting the following crashes currently:

• A majority of main process crashes coming from Thunderbird (example). This is because Thunderbird uses the mozglue blocklist instead of the launcher process blocklist. In my tests, Thunderbird is able to block the injection of my own Database.dll while it is running, so this does not appear to be a bug with the blocklist. Instead, it is likely that the DLL from SolidWorks gets loaded in the main process before we reach the mozglue blocklist initialization, and so the block does not happen because our hooks are not present yet.

• A few main process crashes in Firefox Release (example) and ESR (example). These crashes show launcherProcessState: 1 (FailDisabled) in their telemetry environment, so Firefox used the mozglue blocklist as a fallback. So the presence of these crashes is explained by the same reasons as the Thunderbird crashes.

• A few utility process crashes in Firefox Nightly (example). These crashes show launcherProcessState: 0 (Enabled) in their telemetry environment. They are present because our blocking of Database.dll is only for the main process and the recent work on the out-of-process file picker means that this crash now happens in a utility process on Nightly (see mozilla::detail::fd_async::details::GetStrategy).

It should be fairly easy to solve the utility process crashes by extending the block to utility processes, and we should do that before letting the out-of-process file picker ride the trains.

Blocking the DLL with the mozglue blocklist seems harder though and will probably require more thinking and maybe debugging. We could also consider using the launcher process in Thunderbird I guess; I don't know the details here regarding why it is not being used at the moment.

Comment 1

[Ray Kraesig [:rkraesig]]

(In reply to Yannis Juglaret [:yannis] from comment #0)

It should be fairly easy to solve the utility process crashes by extending the block to utility processes, and we should do that before letting the out-of-process file picker ride the trains.

This has been carved off into bug 1907409, hopefully leaving this to be a purely [meta] bug.