Crash in msmpeg2vdec.dll@0x25a3fb correlated to Trend Micro

RESOLVED WORKSFORME

Status

defect
--
critical
RESOLVED WORKSFORME
11 months ago
5 months ago

People

(Reporter: philipp, Unassigned)

Tracking

(Blocks 1 bug, {crash})

unspecified
x86_64
Windows 7
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr60 affected, firefox62 affected, firefox63- affected)

Details

(Whiteboard: [AV:Trend Micro], crash signature)

This bug was filed from the Socorro interface and is
report bp-2c5302fa-efbd-49dc-a7a6-77e3a0180907.
=============================================================

Top 2 frames of crashing thread:

0 msmpeg2vdec.dll msmpeg2vdec.dll@0x25a3fb 
1 ntdll.dll RtlpLookupDynamicFunctionEntry 

=============================================================

this crash signature is spiking up since 2018-09-06 with 64bit builds of firefox on windows 7. user comments are generally indicating that this is a repeat crash hitting every minute or so, so it's having a particularly bad impact on affected users.  

correlations show that this is fully correlated to modules from Trend Micro Security software, so likely a recent signature update of them is causing this instability:
(100.0% in signature vs 00.99% overall) Module "TmUmEvt64.dll" = true
(100.0% in signature vs 00.99% overall) Module "tmmon64.dll" = true

the crashing trend with a low on weekends is indicating that it's primarily affecting users in an enterprise environment and we also got this report on sumo about it: https://support.mozilla.org/en-US/questions/1233453
Marco, would you mind reaching out?
Flags: needinfo?(mcastelluccio)
Blocks: 1435797
Flags: needinfo?(mcastelluccio)
Whiteboard: [AV:Trend Micro]
I've reached out to the only contact I have. I see Makoto and Ryan were in contact with Trend Micro in bug 1196899, maybe they can help reaching out again?
Flags: needinfo?(ryanvm)
Flags: needinfo?(m_kato)
I've emailed my contacts and CCed you.
Flags: needinfo?(ryanvm)
Flags: needinfo?(m_kato)
The number of crashes on beta were divided by 3 on both beta and release over the last 3 weeks and the last crash with Nightly 64 was also 3 weeks ago. I suppose that an update to Trend Micro fixed the issue upstream and that the numbers will keep on improving as people update their security software. Untracking for 63 as a consequence but leaving the 63 status as affected to keep an eye on the 63 crashes in the coming weeks.
unfortunately the recent numbers don't look like the issue has received a fix from trend micro yet. should we poke them about the current state again?
(In reply to [:philipp] from comment #5)
> unfortunately the recent numbers don't look like the issue has received a
> fix from trend micro yet. should we poke them about the current state again?

Trend Micro was able to get logs to investigate the crash and is working on a hotfix that should be released in November.
Trend Micro has released the hotfix: https://esupport.trendmicro.com/en-us/home/pages/technical-support/1121641.aspx.
More details about the issue in the hotfix readme at https://esupport.trendmicro.com/media/14013962/Ti_150_win_en_amsp_hfb8.10.1054_readme.txt:
> The "TmUmEvt.dll" library should stop stack tracing once the
> "msmpeg2vdec.dll" library loads. However, the function that checks
> if the library has loaded will not work before AMSP starts, which
> may cause which may cause Mozilla(R) Firefox(R) to crash if it
> starts before Trend Micro Security Software gets ready.

Closing because no crashes reported for 12 weeks.

Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.